The Annals of Statistics

Frequentist optimality of Bayesian wavelet shrinkage rules for Gaussian and non-Gaussian noise

Marianna Pensky

Full-text: Open access

Abstract

The present paper investigates theoretical performance of various Bayesian wavelet shrinkage rules in a nonparametric regression model with i.i.d. errors which are not necessarily normally distributed. The main purpose is comparison of various Bayesian models in terms of their frequentist asymptotic optimality in Sobolev and Besov spaces.

We establish a relationship between hyperparameters, verify that the majority of Bayesian models studied so far achieve theoretical optimality, state which Bayesian models cannot achieve optimal convergence rate and explain why it happens.

Article information

Source
Ann. Statist., Volume 34, Number 2 (2006), 769-807.

Dates
First available in Project Euclid: 27 June 2006

Permanent link to this document
https://projecteuclid.org/euclid.aos/1151418240

Digital Object Identifier
doi:10.1214/009053606000000128

Mathematical Reviews number (MathSciNet)
MR2283392

Zentralblatt MATH identifier
1095.62049

Subjects
Primary: 62G08: Nonparametric regression
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures

Keywords
Bayesian models optimality Sobolev and Besov spaces nonparametric regression wavelet shrinkage

Citation

Pensky, Marianna. Frequentist optimality of Bayesian wavelet shrinkage rules for Gaussian and non-Gaussian noise. Ann. Statist. 34 (2006), no. 2, 769--807. doi:10.1214/009053606000000128. https://projecteuclid.org/euclid.aos/1151418240


Export citation

References

  • Abramovich, F., Amato, U. and Angelini, C. (2004). On optimality of Bayesian wavelet estimators. Scand. J. Statist. 31 217–234.
  • Abramovich, F., Besbeas, P. and Sapatinas, T. (2002). Empirical Bayes approach to block wavelet function estimation. Comput. Statist. Data Anal. 39 435–451.
  • Abramovich, F., Sapatinas, T. and Silverman, B. W. (1998). Wavelet thresholding via a Bayesian approach. J. R. Stat. Soc. Ser. B Stat. Methodol. 60 725–749.
  • Angelini, C. and Sapatinas, T. (2004). Empirical Bayes approach to wavelet regression using $\varepsilon$-contaminated priors. J. Stat. Comput. Simul. 74 741–764.
  • Autin, F., Picard, D. and Rivoirard, V. (2004). Maxiset comparisons of procedures, application to choosing priors in a Bayesian nonparametric setting. Unpublished manuscript. Available at www.proba.jussieu.fr/mathdoc/textes/PMA-931.pdf.
  • Barber, S., Nason, G. P. and Silverman, B. W. (2002). Posterior probability intervals for wavelet thresholding. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 189–205.
  • Chipman, H. A., Kolaczyk, E. D. and McCulloch, R. E. (1997). Adaptive Bayesian wavelet shrinkage. J. Amer. Statist. Assoc. 92 1413–1421.
  • Clyde, M. and George, E. I. (2000). Flexible empirical Bayes estimation for wavelets. J. R. Stat. Soc. Ser. B Stat. Methodol. 62 681–698.
  • Clyde, M., Parmigiani, G. and Vidakovic, B. (1996). Bayesian strategies for wavelet analysis. Statist. Comput. Graph. Newsletter 7 2.
  • Clyde, M., Parmigiani, G. and Vidakovic, B. (1998). Multiple shrinkage and subset selection in wavelets. Biometrika 85 391–401.
  • Daubechies, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia.
  • Donoho, D. L. and Johnstone, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 425–455.
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Statist. 24 508–539.
  • Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879–921.
  • Gradshteyn, I. S. and Ryzhik, I. M. (1980). Tables of Integrals, Series, and Products. Academic Press, New York.
  • Johnstone, I. M. (2004). Function Estimation and Gaussian Sequence Models. Unpublished monograph. Available at www-stat.stanford.edu/~imj/.
  • Johnstone, I. M. and Silverman, B. W. (2004). Finding needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences. Ann. Statist. 32 1594–1649.
  • Johnstone, I. M. and Silverman, B. W. (2004). Boundary coiflets for wavelet shrinkage in function estimation. J. Appl. Probab. 41A 81–98.
  • Johnstone, I. M. and Silverman, B. W. (2005). Empirical Bayes selection of wavelet thresholds. Ann. Statist. 33 1700–1752.
  • Mallat, S. G. (1989). A theory for multiresolution signal decomposition: The wavelet representation. IEEE Trans. Pattern Analysis Machine Intelligence 11 674–693.
  • Neumann, M. H. and von Sachs, R. (1995). Wavelet thresholding: Beyond the Gaussian i.i.d. situation. Wavelets and Statistics. Lecture Notes in Statist. 103 301–329. Springer, New York.
  • Shiryaev, A. N. (1996). Probability, 2nd ed. Springer, New York.
  • Simoncelli, E. P. (1997). Statistical models for images: Compression, restoration and synthesis. In Proc. 31st Asilomar Conference on Signals, Systems and Computers 673–678. IEEE Comput. Soc., Pacific Grove, CA.
  • Vidakovic, B. (1998). Nonlinear wavelet shrinkage with Bayes rules and Bayes factors. J. Amer. Statist. Assoc. 93 173–179.
  • Vidakovic, B. (1999). Statistical Modeling by Wavelets. Wiley, New York.
  • Vidakovic, B. and Ruggeri, F. (2001). BAMS method: Theory and simulations. Sankhyā Ser. B 63 234–249.