This article provides a review of Bayesian model averaging as a means of optimizing the predictive performance of common statistical models applied to large-scale educational assessments. The Bayesian framework recognizes that in addition to parameter uncertainty, there is uncertainty in the choice of models themselves. A Bayesian approach to addressing the problem of model uncertainty is the method of Bayesian model averaging. Bayesian model averaging searches the space of possible models for a set of submodels that satisfy certain scientific principles and then averages the coefficients across these submodels weighted by each model's posterior model probability (PMP). Using the weighted coefficients for prediction has been shown to yield optimal predictive performance according to certain scoring rules. We demonstrate the utility of Bayesian model averaging for prediction in education research with three examples: Bayesian regression analysis, Bayesian logistic regression, and a recently developed approach for Bayesian structural equation modeling. In each case, the model-averaged estimates are shown to yield better prediction of the outcome of interest than any submodel based on predictive coverage and the log-score rule. Implications for the design of large-scale assessments when the goal is optimal prediction in a policy context are discussed.
Keywords: Bayesian model averaging; education; large-scale assessments.
Kaplan D, Chen J. Kaplan D, et al. Multivariate Behav Res. 2014 Nov-Dec;49(6):505-17. doi: 10.1080/00273171.2014.928492. Multivariate Behav Res. 2014. PMID: 26735355 Free PMC article.
Bergh DVD, Clyde MA, Gupta ARKN, de Jong T, Gronau QF, Marsman M, Ly A, Wagenmakers EJ. Bergh DVD, et al. Behav Res Methods. 2021 Dec;53(6):2351-2371. doi: 10.3758/s13428-021-01552-2. Epub 2021 Apr 9. Behav Res Methods. 2021. PMID: 33835394 Free PMC article.
Kaplan D. Kaplan D. Psychometrika. 2021 Mar;86(1):215-238. doi: 10.1007/s11336-021-09754-5. Epub 2021 Mar 15. Psychometrika. 2021. PMID: 33721184 Free PMC article.
Wu Y, Hoffman FO, Apostoaei AI, Kwon D, Thomas BA, Glass R, Zablotska LB. Wu Y, et al. Environ Health. 2019 Apr 8;18(1):31. doi: 10.1186/s12940-019-0468-4. Environ Health. 2019. PMID: 30961632 Free PMC article. Review.
Miaou SP, Song JJ. Miaou SP, et al. Accid Anal Prev. 2005 Jul;37(4):699-720. doi: 10.1016/j.aap.2005.03.012. Epub 2005 Apr 12. Accid Anal Prev. 2005. PMID: 15949462 Review.
Bahrami S, Hajian-Tilaki K, Bayani M, Chehrazi M, Mohamadi-Pirouz Z, Amoozadeh A. Bahrami S, et al. BMC Med Res Methodol. 2023 Jul 6;23(1):163. doi: 10.1186/s12874-023-01981-x. BMC Med Res Methodol. 2023. PMID: 37415112 Free PMC article.
Robitzsch A. Robitzsch A. Entropy (Basel). 2022 May 27;24(6):760. doi: 10.3390/e24060760. Entropy (Basel). 2022. PMID: 35741481 Free PMC article.