Bayesian consistency, or (again) lack thereof
This is a placeholder for some discussion about the circumstances (and the manner) in which Bayesian posterior updates to a given prior will converge to the “true” parameter value. In applied math, the model is almost always mis-specified, and so we have to ask “if the truth is complicated and I apply Bayesian procedures to my simplified model, what kind of mistakes am I likely (or not likely) to make?”
“Asymptotically we are all dead (Thoughts about the Bernstein-von Mises theorem before and after a Diamanda Galás concert)” by Dan Simpson (Nov 27, 2017)
“Circling back to an old Bayesian “counterexample” and “On that example of Robins and Ritov; or A sleeping dog in harbor is safe, but that’s not what sleeping dogs are for” by Dan Simpson (Nov 15, 2022)
“Statistical methods that work in some settings but not others,” by Andrew Gelman (Nov 3, 2012)
“Frequentist Consistency of Bayesian Procedures,” a Bactra notebook.
“Bayesian consistency,” a post by The Leisure of the Theory Class blog.
"Limiting Behavior of Posterior Distributions when the Model is Incorrect." Robert H Berk. Ann. Math. Statist. Vol 37 No 1, pgs 51 - 58. (1966)
"Consistency a Posteriori." Robert H Berk, Ann. Math. Statist. Vol 41 No 3, pgs 894 - 906. (1970)
"On the Consistency of Bayes Estimates," Persi Diaconis. David Freedman. Ann. Statist. Vol 14 No 1, pgs 1 - 26. (1986)
“Wald Lecture: On the Bernstein-von Mises theorem with infinite-dimensional parameters,” by David Freedman. Ann. Statist. Vol 27, No 4, pgs 1119-1141. (1999)
“The consistency of posterior distributions in nonparametric problems,” by Andrew Barron, Mark J Schervish, and Larry Wasserman. Annals of Statistics, Vol 27 No 2, pgs 536-561. (1999)
“Freedman’s Neglected Theorem,” by Larry Wasserman on the result that “‘almost all’ Bayesian prior distributions yield inconsistent posteriors.
“Misspecification in infinite-dimensional Bayesian statistics,” by BJK Kleijn and AW Van der Vaart. Annals of Statistics, Vol 34 No 2, pgs 837-877. (2006)
“Bayesian inconsistency under misspecification,” by Peter D Grunwald.
“Bayesian consistency for Markov processes,” by Subhashis Ghosal and Yongqiang Tang. Sankhya: The Indian Journal of Statistics. Vol 68 No 2, pgs 227-239. (2006)
