Electronic Journal of Statistics

Concentration rate and consistency of the posterior distribution for selected priors under monotonicity constraints

Jean-Bernard Salomond

Full-text: Open access

Abstract

In this paper, we consider the well known problem of estimating a density function under qualitative assumptions. More precisely, we estimate monotone non-increasing densities in a Bayesian setting and derive concentration rate for the posterior distribution for a Dirichlet process and finite mixture prior. We prove that the posterior distribution based on both priors concentrates at the rate $(n/\log(n))^{-1/3}$, which is the minimax rate of estimation up to a $\log(n)$ factor. We also study the behaviour of the posterior for the point-wise loss at any fixed point of the support of the density and for the sup-norm. We prove that the posterior distribution is consistent for both loss functions.

Article information

Source
Electron. J. Statist., Volume 8, Number 1 (2014), 1380-1404.

Dates
First available in Project Euclid: 20 August 2014

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1408540291

Digital Object Identifier
doi:10.1214/14-EJS929

Mathematical Reviews number (MathSciNet)
MR3263126

Zentralblatt MATH identifier
1298.62064

Subjects
Primary: 62G07: Density estimation 62G20: Asymptotic properties

Keywords
Density estimation Bayesian inference concentration rate

Citation

Salomond, Jean-Bernard. Concentration rate and consistency of the posterior distribution for selected priors under monotonicity constraints. Electron. J. Statist. 8 (2014), no. 1, 1380--1404. doi:10.1214/14-EJS929. https://projecteuclid.org/euclid.ejs/1408540291


Export citation

References

  • Balabdaoui, F. and Wellner, J. A. (2007). Estimation of a $k$-monotone density: limit distribution theory and the spline connection. Ann. Statist., 35(6):2536–2564.
  • Brunner, L. J. and Lo, A. Y. (1989). Bayes methods for a symmetric unimodal density and its mode. Ann. Statist., 17(4):1550–1566.
  • Castillo, I. (2013). On Bayesian supremum norm contraction rates. arXiv preprint arXiv:1304.1761.
  • Donoho, D. L. and Liu, R. C. (1991). Geometrizing rates of convergence, ii. The Annals of Statistics, pages 633–667.
  • Durot, C., Kulikov, V. N., and Lopuhaä, H. P. (2012). The limit distribution of the $L\sb \infty$-error of Grenander-type estimators. Ann. Statist., 40(3):1578–1608.
  • Ferguson, T. S. (1983). Bayesian density estimation by mixtures of normal distributions. In Recent Advances in Statistics, pages 287–302. Academic Press, New York.
  • Ghosal, S., Ghosh, J. K., and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist., 28(2):500–531.
  • Ghosal, S. and van der Vaart, A. (2007). Posterior convergence rates of Dirichlet mixtures at smooth densities. Ann. Statist., 35(2):697–723.
  • Ghosal, S. and van der Vaart, A. W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist., 29(5):1233–1263.
  • Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist., 38(2):1122–1170.
  • Grenander, U. (1956). On the theory of mortality measurement. II. Skand. Aktuarietidskr., 39:125–153 (1957).
  • Groeneboom, P. (1985). Estimating a monotone density. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, Vol. II (Berkeley, Calif., 1983), Wadsworth Statist./Probab. Ser., pages 539–555. Wadsworth, Belmont, CA.
  • Groeneboom, P. (1986). Some current developments in density estimation. In Mathematics and Computer Science (Amsterdam, 1983), volume 1 of CWI Monogr., pages 163–192. North-Holland, Amsterdam.
  • Hoffmann, M., Rousseau, J., and Schmidt-Hieber, J. (2013). On adaptive posterior concentration rates. arXiv preprint arXiv:1305.5270.
  • Khazaei, S., Rousseau, J., and Balabdaoui, F. (2010). Bayesian Nonparametric Inference of decreasing densities. In 42èmes Journées de Statistique. Marseille, France.
  • Kruijer, W., Rousseau, J., Van Der Vaart, A., et al. (2010). Adaptive bayesian density estimation with location-scale mixtures. Technical report.
  • Le Cam, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer Series in Statistics. Springer-Verlag, New York.
  • Lo, A. Y. (1984). On a class of Bayesian nonparametric estimates. I. Density estimates. Ann. Statist., 12(1):351–357.
  • Prakasa Rao, B. L. S. (1970). Estimation for distributions with monotone failure rate. Ann. Math. Statist., 41:507–519.
  • Reynaud-Bouret, P., Rivoirard, V., and Tuleau-Malot, C. (2011). Adaptive density estimation: a curse of support? J. Statist. Plann. Inference, 141(1):115–139.
  • Rivoirard, V. and Rousseau, J. (2012). Bernstein-von Mises theorem for linear functionals of the density. Ann. Statist., 40(3):1489–1523.
  • Robertson, T., Wright, F. T., and Dykstra, R. L. (1988). Order Restricted Statistical Inference. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley & Sons, Ltd., Chichester.
  • Rousseau, J. (2010). Rates of convergence for the posterior distributions of mixtures of betas and adaptive nonparametric estimation of the density. Ann. Statist., 38(1):146–180.
  • Rousseau, J. and Mengersen, K. (2011). Asymptotic behaviour of the posterior distribution in overfitted mixture models. J. R. Stat. Soc. Ser. B Stat. Methodol., 73(5):689–710.
  • Shen, W., Tokdar, S. T., and Ghosal, S. (2013). Adaptive Bayesian multivariate density estimation with Dirichlet mixtures. Biometrika, 100(3):623–640.
  • Sun, J. and Woodroofe, M. (1996). Adaptive smoothing for a penalized NPMLE of a non-increasing density. J. Statist. Plann. Inference, 52(2):143–159.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak convergence and Empirical Processes. Springer Series in Statistics. Springer-Verlag, New York. With applications to statistics.
  • Williamson, R. E. (1956). Multiply monotone functions and their Laplace transforms. Duke Math. J., 23:189–207.
  • Wu, Y. and Ghosal, S. (2008). Kullback Leibler property of kernel mixture priors in Bayesian density estimation. Electron. J. Stat., 2:298–331.