Bayesian methods in statistics – article review –

Authors

  • Najlaa Saad Ibrahim Department of Statistics and Informatics, College of Computer Science and Mathematics, University of Mosul Author
  • Sura Mohamed jamal alden Department of Statistics and Informatics, College of Computer Science and Mathematics, University of Mosul Author

DOI:

https://doi.org/10.62933/kqy88t52

Keywords:

Bayesian inference, Posterior distribution, Markov Chain Monte Carlo (MCMC), Gibbs Sampling, Hamiltonian Monte Carlo, No-U-Turn Sampler

Abstract

    A basic substitute for conventional iterative models, Bayesian statistics gives a thorough intellectual framework. Instead of only reflecting the proportional frequency of events, this method redefines probability as a statement of individual certainty. The philosophical underpinnings of Bayesian statistics are first examined in this methodical research, with a focus on the cognitive updating process. In order to create updated views expressed through subsequent distributions, this approach relies on the methodical integration of previous beliefs (expressed through initial distributions) with fresh data obtained from real-world experiments (using a probability function). In sophisticated computations, Bayesian statistics presents difficulties despite its many benefits. However, these difficulties have been largely overcome because to developments in simulation methods like the Markov Monte Carlo Chain (MCMC). The Metropolis-Hastings algorithm, Gibbs sampling, Monte Carlo Hamiltonian, and the Nuclear-Inverted Sampling (NUTS) technique are the four main algorithms that are examined in this paper within the MCMC framework. It explains how they developed historically and examines their theoretical and practical traits. The study compares the benefits of the Bayesian approach, such as model design flexibility and uncertainty management, with its drawbacks, such as initial distribution selection and computational complexity. In view of the increased usage of artificial intelligence technologies and large data, the study also examines the prospects for the Bayesian approach in the future. In order to help researchers and professionals in the field stay up to date with the latest advancements in this quickly growing field, the review finishes with practical advice. These recommendations include identifying interesting research directions.

References

[1] Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2013). “Bayesian Data Analysis (3rd ed.)”. Chapman and Hall/CRC. https://doi.org/10.1201/b16018

[2] Bernardo, J. M., & Smith, A. F. M. (1994). “Bayesian Theory”. JOHN WILEY & SONS, LTD. http://www.wiley.co.uk or http://www.wilcy.com

[3] Oqbah , M.A. & Al_Rassam , R.S. (2025).”Bayesian Estimation of Stress-Strength Reliability Function P(T

[4] Howson, C., & Urbach, P. (2006). "Scientific Reasoning: The Bayesian Approach (3rd ed.)". Open Court Publishing. http://fitelson.org/probability/howson_and_urbach_3rd_ed.pdf

[5] Kruschke, J. K. (2014). “Doing Bayesian Data Analysis”. A Tutorial with R, JAGS, and Stan (2nd ed.). http://dx.doi.org/10.1016/B978-0-12-405888-0.00001-5

[6] van de Schoot, R., Depaoli, S., King, R. & Kramer, B., (2021). “Bayesian statistics and modelling”. Nature Reviews Methods Primers, 1(1), 1-26. DOI:10.1038/s43586-020-00001-2

[7] Brooks, S., Gelman, A., Jones, G., & Meng, X. L. (2011). “Handbook of Markov Chain Monte Carlo”. Chapman and Hall/CRC. https://doi.org/10.1201/b10905

[8] McGrayne, S. B. (2013). “The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy”. The European Legacy, 18(7), pp. 965–966. DOI:10.1080/10848770.2013.817786

[9] Efron, B. (2013). “Bayes' Theorem in the 21st Century”. Science, 340(6137), 1177-1178. DOI:10.1126/science.1236536

[10] Chib, S. & Greenberg, E., (1995).” Understanding the Metropolis-Hastings Algorithm”, The American Statistician, Vol. 49, No.4, pp. 327-335.http://links.jstor.org/sici?sici=0003-1305%28199511%2949%3A4%3C327%3AUTMA%3E2.0.CO%3B2-2.

[11] Ahmad, H. A. & Saieed, H. A.,(2024),” Bayesian Estimation of TLR distribution under Sq and E linex loss”. Iraqi Journal of Statistical Sciences, Vol. 21, No. 2, pp.64-71 http://creativecommons.org/licenses/by/4.0/

[12] Casella,G. & George, E.,(1992) ”Explaining the Gibbs Sampler”, The American Statistician, vol. 46, No. 3, 1992. http://www.jstor.org/

[13] Betancourt, M., (2017).”A Conceptual Introduction to Hamiltonian Monte Carlo”. arXiv:1701.02434. https://doi.org/10.48550/arXiv.1701.02434

[14] Neal, R., (2012).” MCMC using Hamiltonian Dynamics”. Handbook of Markov Chain Monte Carlo. DOI:10.1201/b10905-6

[15] Green, P. J.,(1995).”Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination”. Biometrika, 82(4):711–732. https://doi.org/10.1093/biomet/82.4.711

[16] Matthew, D. H. and Andrew, G., (2014).” The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo”. Journal of Machine Learning Research, no.15, PP.1593-1623. https://www.jmlr.org/papers/volume15/hoffman14a/hoffman14a.pdf

[17] Schneider, M., Grahl, J., Francas, D. & Vigo, D., (2013).” A problem-adjusted genetic algorithm for flexibility design”. International Journal of Production Economics, 141(1). https://doi.org/10.1016/j.ijpe.2012.05.017

[18] Hoffman, M. D., & Gelman, A. (2014). "The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo". Journal of Machine Learning Research, 15(47), 1593-1623. https://www.jmlr.org/papers/volume15/hoffman14a/hoffman14a.pdf

[19] Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M. A., Guo, J., Li, P., & Riddell, A. (2017). Stan: A Probabilistic Programming Language. Journal of Statistical Software, 76(1), 1-32. doi: 10.18637/jss.v076.i01

[20] Geyer, C. J. (2011). Introduction to Markov Chain Monte Carlo. In S. Brooks, A. Gelman, G. Jones, & X. L. Meng (Eds.), Handbook of Markov Chain Monte Carlo (pp. 3-48). Boca Raton: CRC Press. https://scispace.com/pdf/introduction-to-markov-chain-monte-carlo-4i4ybfwvbi.pdf

[21] Robert, C. P., & Casella, G. (2010). Monte Carlo Statistical Methods (2nd ed.). New York: Springer. https://mcube.lab.nycu.edu.tw/~cfung/docs/books/robert2004monte_carlo_statistical_methods.pdf

Comparison between the four algorithms.

Downloads

Published

2026-07-29

Issue

Section

Review Articles

How to Cite

Bayesian methods in statistics – article review –. (2026). Iraqi Statisticians Journal, 3(2), 67-76. https://doi.org/10.62933/kqy88t52