Hamiltonian Monte Carlo Methods in Machine Learning
- 1st Edition - February 3, 2023
- Imprint: Academic Press
- Authors: Tshilidzi Marwala, Rendani Mbuvha, Wilson Tsakane Mongwe
- Language: English
- Paperback ISBN:9 7 8 - 0 - 4 4 3 - 1 9 0 3 5 - 3
- eBook ISBN:9 7 8 - 0 - 4 4 3 - 1 9 0 3 6 - 0
Hamiltonian Monte Carlo Methods in Machine Learning introduces methods for optimal tuning of HMC parameters, along with an introduction of Shadow and Non-canonical HMC methods w… Read more
Purchase options
Institutional subscription on ScienceDirect
Request a sales quoteHamiltonian Monte Carlo Methods in Machine Learning introduces methods for optimal tuning of HMC parameters, along with an introduction of Shadow and Non-canonical HMC methods with improvements and speedup. Lastly, the authors address the critical issues of variance reduction for parameter estimates of numerous HMC based samplers. The book offers a comprehensive introduction to Hamiltonian Monte Carlo methods and provides a cutting-edge exposition of the current pathologies of HMC-based methods in both tuning, scaling and sampling complex real-world posteriors. These are mainly in the scaling of inference (e.g., Deep Neural Networks), tuning of performance-sensitive sampling parameters and high sample autocorrelation.
Other sections provide numerous solutions to potential pitfalls, presenting advanced HMC methods with applications in renewable energy, finance and image classification for biomedical applications. Readers will get acquainted with both HMC sampling theory and algorithm implementation.
- Provides in-depth analysis for conducting optimal tuning of Hamiltonian Monte Carlo (HMC) parameters
- Presents readers with an introduction and improvements on Shadow HMC methods as well as non-canonical HMC methods
- Demonstrates how to perform variance reduction for numerous HMC-based samplers
- Includes source code from applications and algorithms
Researchers, developers, and industry professionals in Artificial Intelligence and Statistics, as well as Computational Biology, Bioinformatics, Computational Modelling, and Biomedical Modelling as well as researchers and industry professionals in Information Technology and Computer Science, such as developers of AI, Machine Learning, and Deep Learning
- Cover image
- Title page
- Table of Contents
- Copyright
- Dedication
- List of figures
- List of tables
- Authors
- Tshilidzi Marwala
- Wilson Tsakane Mongwe
- Rendani Mbuvha
- Foreword
- Preface
- References
- Nomenclature
- List of symbols
- 1: Introduction to Hamiltonian Monte Carlo
- Abstract
- 1.1. Introduction
- 1.2. Background to Markov Chain Monte Carlo
- 1.3. Metropolis-Hastings algorithm
- 1.4. Metropolis Adjusted Langevin algorithm
- 1.5. Hamiltonian Monte Carlo
- 1.6. Magnetic Hamiltonian Monte Carlo
- 1.7. Quantum-Inspired Hamiltonian Monte Carlo
- 1.8. Separable Shadow Hamiltonian Hybrid Monte Carlo
- 1.9. No-U-Turn Sampler algorithm
- 1.10. Antithetic Hamiltonian Monte Carlo
- 1.11. Book objectives
- 1.12. Book contributions
- 1.13. Conclusion
- References
- 2: Sampling benchmarks and performance metrics
- Abstract
- 2.1. Benchmark problems and datasets
- 2.2. Performance metrics
- 2.3. Algorithm parameter tuning
- 2.4. Conclusion
- References
- 3: Stochastic volatility Metropolis-Hastings
- Abstract
- 3.1. Proposed methods
- 3.2. Experiments
- 3.3. Results and discussion
- 3.4. Conclusion
- References
- 4: Quantum-inspired magnetic Hamiltonian Monte Carlo
- Abstract
- 4.1. Proposed algorithm
- 4.2. Experiment description
- 4.3. Results and discussion
- 4.4. Conclusion
- References
- 5: Generalised magnetic and shadow Hamiltonian Monte Carlo
- Abstract
- 5.1. Proposed partial momentum retention algorithms
- 5.2. Experiment description
- 5.3. Results and discussion
- 5.4. Conclusion
- References
- 6: Shadow Magnetic Hamiltonian Monte Carlo
- Abstract
- 6.1. Background
- 6.2. Shadow Hamiltonian for MHMC
- 6.3. Proposed Shadow Magnetic algorithm
- 6.4. Experiment description
- 6.5. Results and discussion
- 6.6. Conclusion
- References
- 7: Adaptive Shadow Hamiltonian Monte Carlo
- Abstract
- 7.1. Proposed adaptive shadow algorithm
- 7.2. Experiment description
- 7.3. Results and discussion
- 7.4. Conclusion
- References
- 8: Adaptive noncanonical Hamiltonian Monte Carlo
- Abstract
- 8.1. Background
- 8.2. Proposed algorithm
- 8.3. Experiments
- 8.4. Results and discussion
- 8.5. Conclusions
- References
- 9: Antithetic Hamiltonian Monte Carlo techniques
- Abstract
- 9.1. Proposed antithetic samplers
- 9.2. Experiment description
- 9.3. Results and discussion
- 9.4. Conclusion
- References
- 10: Bayesian neural network inference in wind speed nowcasting
- Abstract
- 10.1. Background
- 10.2. Experiment setup
- 10.3. Results and discussion
- 10.4. Conclusion
- References
- 11: A Bayesian analysis of the efficacy of Covid-19 lockdown measures
- Abstract
- 11.1. Background
- 11.2. Methods
- 11.3. Results and discussion
- 11.4. Conclusion
- References
- 12: Probabilistic inference of equity option prices under jump-diffusion processes
- Abstract
- 12.1. Background
- 12.2. Numerical experiments
- 12.3. Results and discussions
- 12.4. Conclusion
- References
- 13: Bayesian inference of local government audit outcomes
- Abstract
- 13.1. Background
- 13.2. Experiment description
- 13.3. Results and discussion
- 13.4. Conclusion
- References
- 14: Conclusions
- Abstract
- 14.1. Summary of contributions
- 14.2. Ongoing and future work
- References
- A: Separable shadow Hamiltonian
- A.1. Derivation of separable shadow Hamiltonian
- A.2. S2HMC satisfies detailed balance
- A.3. Derivatives from non-canonical Poisson brackets
- References
- B: ARD posterior variances
- C: ARD committee feature selection
- D: Summary of audit outcome literature survey
- References
- References
- References
- Index
- Edition: 1
- Published: February 3, 2023
- No. of pages (Paperback): 220
- No. of pages (eBook): 220
- Imprint: Academic Press
- Language: English
- Paperback ISBN: 9780443190353
- eBook ISBN: 9780443190360
TM
Tshilidzi Marwala
Dr. Tshilidzi Marwala is the Rector of the United Nations (UN) University and the UN Under-Secretary-General from 1 March 2023. He was previously the Vice-Chancellor and Principal of the University of Johannesburg, Deputy Vice-Chancellor for Research and Executive Dean of the Faculty of Engineering at the University of Johannesburg. He was Associate Professor, Full Professor, the Carl and Emily Fuchs Chair of Systems and Control Engineering at the University of the Witwatersrand. He holds a Bachelor of Science in Mechanical Engineering (magna cum laude) from Case Western Reserve University, a Master of Mechanical Engineering from the University of Pretoria, PhD in Artificial Intelligence from the University of Cambridge and a Post-Doc at Imperial College (London). He is a registered professional engineer, a Fellow of TWAS (The World Academy of Sciences), the Academy of Science of South Africa, the African Academy of Sciences and the South African Academy of Engineering. He is a Senior Member of the IEEE and a distinguished member of the ACM. His research interests are multi-disciplinary and they include the theory and application of artificial intelligence to
engineering, computer science, finance, social science and medicine. He has supervised 28 Doctoral students published 15 books in artificial intelligence (one translated into Chinese), over 300 papers in journals, proceedings, book chapters and magazines and holds five patents. He is an associate editor of the International Journal of Systems Science (Taylor and Francis Publishers). He has been a visiting scholar at Harvard University, University of California at Berkeley, Wolfson College of the University of Cambridge, Nanjing Tech University and Silesian University of Technology in Poland. His opinions have appeared in the New Scientist, The Economist, Time Magazine, BBC, CNN and the Oxford Union. Dr. Marwala is the author of Rational Machines and Artificial Intelligence from Elsevier Academic Press.
Affiliations and expertise
Rector of the United Nations (UN) University and the UN Under-Secretary-General in Tokyo, Japan, from 1 March 2023RM
Rendani Mbuvha
Dr. Rendani Mbuvha is a lecturer in Statistics and Actuarial Science at the University of Witwatersrand, Johannesburg, South Africa. He is a qualified Actuary and a holder of the Chartered Enterprise Risk Actuary designation. He holds a BSc with Honors in Actuarial Science and Statistics from the University of Cape Town, an MSc in Machine Learning from KTH, Royal Institute of Technology in Sweden, and a Ph.D. in Probabilistic Parameter Inference at the University of Johannesburg. He was a recipient of the Google Ph.D. fellowship for his research at the University of Johannesburg. He has previously served in various analytics and actuarial roles in large financial services and AI consulting organizations in both South Africa and Sweden.
Affiliations and expertise
Lecturer in Statistics and Actuarial Science, University of Witwatersrand, Johannesburg, South AfricaWM
Wilson Tsakane Mongwe
Wilson Tsakane Mongwe is a Researcher at the University of Johannesburg, South Africa, specializing in Bayesian machine learning and Markov Chain Monte Carlo methods. He received his BSc in Computing from the University of South Africa, his BBusSci in Actuarial Science from the University of Cape Town, and his MSc in Mathematical Finance from the University of Cape Town. He was the recipient of the Google PhD fellowship in machine learning, which is supporting his PhD research.
Affiliations and expertise
Researcher, University of Johannesburg, South AfricaRead Hamiltonian Monte Carlo Methods in Machine Learning on ScienceDirect