Bayesian Analysis

Bayesian Estimation of the Discrepancy with Misspecified Parametric Models

Pierpaolo De Blasi and Stephen G. Walker

Full-text: Open access


We study a Bayesian model where we have made specific requests about the parameter values to be estimated. The aim is to find the parameter of a parametric family which minimizes a distance to the data generating density and then to estimate the discrepancy using nonparametric methods. We illustrate how coherent updating can proceed given that the standard Bayesian posterior from an unidentifiable model is inappropriate. Our updating is performed using Markov Chain Monte Carlo methods and in particular a novel method for dealing with intractable normalizing constants is required. Illustrations using synthetic data are provided.

Article information

Bayesian Anal., Volume 8, Number 4 (2013), 781-800.

First available in Project Euclid: 4 December 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Asymptotics Bayesian nonparametrics Semi–parametric density model Gaussian process Kullback–Leibler divergence Posterior consistency


De Blasi, Pierpaolo; Walker, Stephen G. Bayesian Estimation of the Discrepancy with Misspecified Parametric Models. Bayesian Anal. 8 (2013), no. 4, 781--800. doi:10.1214/13-BA024.

Export citation


  • Choi, T. and Ramamoorthi, R. V. (2008). “Remarks on consistency of posterior distributions.” In Clarke, B. and Ghosal, S. (eds.), Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, volume 3, 170–186. Institute of Mathematical Statistics.
  • De Blasi, P. and Walker, S. G. (2013). “Bayesian asymptotics with misspecified models.” Statistica Sinica, 23: 169–187.
  • Ghosal, S., Ghosh, J. K., and Ramamoorthi, R. V. (1999). “Posterior consistency of Dirichlet mixtures in density estimation.” The Annals of Statistics, 27: 143–158.
  • Ghosal, S. and Roy, A. (2006). “Posterior consistency of Gaussian process prior for nonparametric binary regression.” The Annals of Statistics, 34: 2413–2429.
  • Ghosal, S. and van der Vaart, A. W. (2007). “Posterior convergence rates of Dirichlet mixtures at smooth densities.” The Annals of Statistics, 35: 697–723.
  • Godsill, S. J. (2001). “On the relationship between Markov chain Monte Carlo methods for model uncertainty.” Journal of Computational and Graphical Statistics, 10: 230–248.
  • Green, P. J. (1995). “Reversible jump Markov chain Monte Carlo computation and Bayesian model determination.” Biometrika, 82: 711–732.
  • Hewitt, E. and Savage, L. J. (1955). “Symmetric measures on Cartesian products.” Transaction of the American Mathematical Society, 80: 470–501.
  • Hjort, N. L. and Glad, I. K. (1995). “Nonparametric density estimation with a parametric start.” The Annals of Statistics, 23: 882–904.
  • Kleijn, B. J. K. and van der Vaart, A. W. (2012). “The Bernstein-Von Mises theorem under misspecification.” Electronic Journal of Statistics, 6: 354–381.
  • Lenk, P. J. (1988). “The logistic normal distribution for Bayesian, nonparametric, predictive densities.” Journal of the American Statistical Association, 83: 509–516.
  • — (2003). “Bayesian semiparametric density estimation and model verification using a logistic Gaussian process.” Journal of Computational and Graphical Statistics, 12: 548–565.
  • Leonard, T. (1978). “Density estimation, stochastic processes and prior information (with discussion).” Journal of the Royal Statistical Society - Series B, 40: 113–146.
  • Liese, F. and Vajda, I. (2006). “On divergence and informations in statistics and information theory.” IEEE Transactions on Information Theory, 52: 4394–4412.
  • Liu, F., Bayarri, M., and Berger, J. O. (2008). “Modularization in Bayesian analysis, with emphasis on analysis of computer models.” Bayesian Analysis, 4: 119–150.
  • Rousseau, J. (2008). “Approximating interval hypothesis: p-values and Bayes factors.” In Bernardo, J. M., Berger, J. O., Dawid, A. P., and Smith, A. F. M. (eds.), Bayesian Statistics, volume 8, 417–452. Oxford University Press.
  • Tokdar, S. (2007). “Towards a faster implementation of density estimation with logistic Gaussian process priors.” Journal of Computational and Graphical Statistics, 16: 633–655.
  • Tokdar, S. and Ghosh, H. J. (2007). “Posterior consistency of Gaussian process priors in density estimation.” Journal of Statistical Planning and Inference, 137: 34–42.
  • van der Vaart, A. W. and van Zanten, J. H. (2008). “Rates of contraction of posterior distributions based on Gaussian process priors.” The Annals of Statistics, 36: 1435–1463.
  • — (2009). “Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth.” The Annals of Statistics, 37: 2655–2675.
  • Verdinelli, I. and Wasserman, L. (1998). “Bayesian goodness–of–fit testing using infinite–dimensional exponential families.” The Annals of Statistics, 26: 1215–1241.
  • Walker, S. G. (2011). “Posterior sampling when the normalizing constant is unknown.” Communications in Statistics - Simulation and Computation, 40: 784–792.
  • — (2013). “Bayesian inference with misspecified models.” Journal of Statistical Planning and Inference.