The Annals of Statistics

Bayesian aspects of some nonparametric problems

Linda H. Zhao

Full-text: Open access


We study the Bayesian approach to nonparametric function estimation problems such as nonparametric regression and signal estimation. We consider the asymptotic properties of Bayes procedures for conjugate (= Gaussian) priors.

We show that so long as the prior puts nonzero measure on the very large parameter set of interest then the Bayes estimators are not satisfactory. More specifically, we show that these estimators do not achieve the correct minimax rate over norm bounded sets in the parameter space. Thus all Bayes estimators for proper Gaussian priors have zero asymptotic efficiency in this minimax sense.

We then present a class of priors whose Bayes procedures attain the optimal minimax rate of convergence. These priors may be viewed as compound, or hierarchical, mixtures of suitable Gaussian distributions.

Article information

Ann. Statist., Volume 28, Number 2 (2000), 532-552.

First available in Project Euclid: 15 March 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation
Secondary: 62A15 62G20: Asymptotic properties

White noise nonparametric regression Bayes minimax conjugate priors


Zhao, Linda H. Bayesian aspects of some nonparametric problems. Ann. Statist. 28 (2000), no. 2, 532--552. doi:10.1214/aos/1016218229.

Export citation


  • Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York.
  • Brown, L. D. (1986). Fundamentals of Statistical Exponential Families with Applications in Statistical Decision Theory. IMS, Hayward, CA.
  • Brown, L. D. and Low, M. G. (1996). Asymptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24 2384-2398.
  • Cox, D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903-923.
  • Diaconis, P. and Freedman, D. (1986). On the consistency of Bayes estimates (with discussion). Ann. Statist. 14 1-67.
  • Diaconis, P. and Freedman, D. (1998). On the Bernstein-von Mises theorem with infinite dimensional parameters. Technical Report 492, Dept. Statistics, Univ. California, Berkeley
  • Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential families. Ann. Statist. 7 269-281.
  • Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879-921.
  • Donoho, D. L. and Liu, R. (1991). Geometrizing rates of convergence III. Ann. Statist. 19 668- 701.
  • Donoho, D. L., Liu, R. and MacGibbon, B. (1990). Minimax risk over hyperrectangles and implications. Ann. Statist. 18 1416-1437.
  • Donoho, D. L. and Low, M. G. (1992). Renormalization exponents and optimal pointwise rates of convergence. Ann. Statist. 20 944-970.
  • Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1 209-230.
  • Freedman, D. (1999). On the Bernstein-von Mises theorem with infinite dimensional parameters. Ann. Statist. 27 1119-1140.
  • Ibragimov, I. A. and Hasminskii, R. Z. (1977). Estimation of infinite-dimensional parameter in white Gaussian noise. Dokl. Akad. Nauk SSSR 236 1053-1056.
  • Klemel¨a, J. and Nussbaum, M. (1998). Constructive asymptotic equivalence of density estimation and Gaussian white noise. Unpublished manuscript.
  • Kuo, H. (1975). Gaussian Measures in Banach Spaces. Lecture Notes in Math. 403. Springer, New York.
  • LeCam, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York.
  • Mandelbaum, A. (1983). Linear estimators of the mean of a Gaussian distribution on a Hilbert space. Ph. D. dissertation, Cornell Univ.
  • Mandelbaum, A. (1984). All admissible linear estimators of the mean of a Gaussian distribution on a Hilbert space. Ann. Statist. 12 1448-1466.
  • Nussbaum, M. (1996). Asymptotic equivalence of density estimation and white noise. Ann. Statist. 24 2399-2430.
  • Pinsker, M. S. (1980). Optimal filtering of square integrable signals in Gaussian white noise. Problemy Peredachi Informatsii 16 52-68 (in Russian). (Translation in Problems Imform. Transmission 16 120-133.)
  • Shen, X. and Wasserman, L. (1998). Rates of convergence of posterior distributions. Unpublished manuscript.
  • Van der Linde, A. (1993). A note on smoothing splines as Bayesian estimates. Statist. Decisions 11 61-67.
  • Van der Linde, A. (1995). Splines from a Bayesian point of view. Test 4 63-81.
  • Wahba, G. (1978). Improper priors, spline smoothing and the problem of guarding against model errors in regression. J. Roy. Statist. Soc. Ser. B 40 364-372.
  • Wahba, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia.
  • Zhao, L. H. (1993). Frequentist and Bayesian Aspects of some nonparametric estimation problems. Ph.D. dissertation, Cornell Univ.
  • Zhao, L. H. (1996). Bayesian aspects of some nonparametric estimation problems (abstract). IMS Bulletin 25 20.