Annals of Statistics

Misspecification in infinite-dimensional Bayesian statistics

B. J. K. Kleijn and A. W. van der Vaart

Full-text: Open access

Abstract

We consider the asymptotic behavior of posterior distributions if the model is misspecified. Given a prior distribution and a random sample from a distribution P0, which may not be in the support of the prior, we show that the posterior concentrates its mass near the points in the support of the prior that minimize the Kullback–Leibler divergence with respect to P0. An entropy condition and a prior-mass condition determine the rate of convergence. The method is applied to several examples, with special interest for infinite-dimensional models. These include Gaussian mixtures, nonparametric regression and parametric models.

Article information

Source
Ann. Statist., Volume 34, Number 2 (2006), 837-877.

Dates
First available in Project Euclid: 27 June 2006

Permanent link to this document
https://projecteuclid.org/euclid.aos/1151418243

Digital Object Identifier
doi:10.1214/009053606000000029

Mathematical Reviews number (MathSciNet)
MR2283395

Zentralblatt MATH identifier
1095.62031

Subjects
Primary: 62G07: Density estimation 62G08: Nonparametric regression 62G20: Asymptotic properties 62F05: Asymptotic properties of tests 62F15: Bayesian inference

Keywords
Misspecification infinite-dimensional model posterior distribution rate of convergence

Citation

Kleijn, B. J. K.; van der Vaart, A. W. Misspecification in infinite-dimensional Bayesian statistics. Ann. Statist. 34 (2006), no. 2, 837--877. doi:10.1214/009053606000000029. https://projecteuclid.org/euclid.aos/1151418243


Export citation

References

  • Berk, R. H. (1966). Limiting behavior of posterior distributions when the model is incorrect. Ann. Math. Statist. \bf37 51–58. [Corrigendum 37 745–746.]
  • Birgé, L. (1983). Approximation dans les espaces métriques et théorie de l'estimation. Z. Wahrsch. Verw. Gebiete 65 181–238.
  • Bunke, O. and Milhaud, X. (1998). Asymptotic behavior of Bayes estimates under possibly incorrect models. Ann. Statist. 26 617–644.
  • Diaconis, P. and Freedman, D. (1986). On the consistency of Bayes estimates (with discussion). Ann. Statist. \bf14 1–67.
  • Diaconis, P. and Freedman, D. (1986). On inconsistent Bayes estimates of location. Ann. Statist. \bf14 68–87.
  • Ferguson, T. S. (1973). A Bayesian analysis of some non-parametric problems. Ann. Statist. \bf1 209–230.
  • Ferguson, T. S. (1974). Prior distributions on spaces of probability measures. Ann. Statist. \bf2 615–629.
  • Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. \bf28 500–531.
  • Ghosal, S. and van der Vaart, A. W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist. \bf29 1233–1263.
  • Le Cam, L. M. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York.
  • Megginson, R. E. (1998). An Introduction to Banach Space Theory. Springer, New York.
  • Pfanzagl, J. (1988). Consistency of maximum likelihood estimators for certain nonparametric families, in particular: Mixtures. J. Statist. Plann. Inference \bf19 137–158.
  • Schwartz, L. (1965). On Bayes procedures. Z. Wahrsch. Verw. Gebiete \bf4 10–26.
  • Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. \bf29 687–714.
  • Strasser, H. (1985). Mathematical Theory of Statistics. de Gruyter, Berlin.
  • Wong, W. H. and Shen, X. (1995). Probability inequalities for likelihood ratios and convergence rates of sieve MLE's. Ann. Statist. \bf23 339–362.