The Annals of Statistics

Frequentist optimality of Bayesian wavelet shrinkage rules for Gaussian and non-Gaussian noise

Marianna Pensky

Full-text: Open access

Enhanced PDF (431 KB)

Abstract

The present paper investigates theoretical performance of various Bayesian wavelet shrinkage rules in a nonparametric regression model with i.i.d. errors which are not necessarily normally distributed. The main purpose is comparison of various Bayesian models in terms of their frequentist asymptotic optimality in Sobolev and Besov spaces.

We establish a relationship between hyperparameters, verify that the majority of Bayesian models studied so far achieve theoretical optimality, state which Bayesian models cannot achieve optimal convergence rate and explain why it happens.

Article information

Source
Ann. Statist., Volume 34, Number 2 (2006), 769-807.

Dates
First available in Project Euclid: 27 June 2006

Permanent link to this document
https://projecteuclid.org/euclid.aos/1151418240

Digital Object Identifier
doi:10.1214/009053606000000128

Mathematical Reviews number (MathSciNet)
MR2283392

Zentralblatt MATH identifier
1095.62049

Subjects
Primary: 62G08: Nonparametric regression
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures

Keywords
Bayesian models optimality Sobolev and Besov spaces nonparametric regression wavelet shrinkage

Citation

Pensky, Marianna. Frequentist optimality of Bayesian wavelet shrinkage rules for Gaussian and non-Gaussian noise. Ann. Statist. 34 (2006), no. 2, 769--807. doi:10.1214/009053606000000128. https://projecteuclid.org/euclid.aos/1151418240


Export citation

References

  • Abramovich, F., Amato, U. and Angelini, C. (2004). On optimality of Bayesian wavelet estimators. Scand. J. Statist. 31 217–234.
    Mathematical Reviews (MathSciNet): MR2066250
    Zentralblatt MATH: 1063.62051
    Digital Object Identifier: doi:10.1111/j.1467-9469.2004.02-087.x
  • Abramovich, F., Besbeas, P. and Sapatinas, T. (2002). Empirical Bayes approach to block wavelet function estimation. Comput. Statist. Data Anal. 39 435–451.
    Mathematical Reviews (MathSciNet): MR1910021
    Zentralblatt MATH: 0993.62032
    Digital Object Identifier: doi:10.1016/S0167-9473(01)00085-8
  • Abramovich, F., Sapatinas, T. and Silverman, B. W. (1998). Wavelet thresholding via a Bayesian approach. J. R. Stat. Soc. Ser. B Stat. Methodol. 60 725–749.
    Mathematical Reviews (MathSciNet): MR1649547
    Zentralblatt MATH: 0910.62031
    Digital Object Identifier: doi:10.1111/1467-9868.00151
  • Angelini, C. and Sapatinas, T. (2004). Empirical Bayes approach to wavelet regression using $\varepsilon$-contaminated priors. J. Stat. Comput. Simul. 74 741–764.
    Mathematical Reviews (MathSciNet): MR2075964
    Zentralblatt MATH: 1052.62006
    Digital Object Identifier: doi:10.1080/00949650310001643162
  • Autin, F., Picard, D. and Rivoirard, V. (2004). Maxiset comparisons of procedures, application to choosing priors in a Bayesian nonparametric setting. Unpublished manuscript. Available at www.proba.jussieu.fr/mathdoc/textes/PMA-931.pdf.
  • Barber, S., Nason, G. P. and Silverman, B. W. (2002). Posterior probability intervals for wavelet thresholding. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 189–205.
    Mathematical Reviews (MathSciNet): MR1904700
    Zentralblatt MATH: 1059.62040
    Digital Object Identifier: doi:10.1111/1467-9868.00332
  • Chipman, H. A., Kolaczyk, E. D. and McCulloch, R. E. (1997). Adaptive Bayesian wavelet shrinkage. J. Amer. Statist. Assoc. 92 1413–1421.
    Zentralblatt MATH: 0913.62027
    Digital Object Identifier: doi:10.1080/01621459.1997.10473662
  • Clyde, M. and George, E. I. (2000). Flexible empirical Bayes estimation for wavelets. J. R. Stat. Soc. Ser. B Stat. Methodol. 62 681–698.
    Mathematical Reviews (MathSciNet): MR1796285
    Zentralblatt MATH: 0957.62006
    Digital Object Identifier: doi:10.1111/1467-9868.00257
  • Clyde, M., Parmigiani, G. and Vidakovic, B. (1996). Bayesian strategies for wavelet analysis. Statist. Comput. Graph. Newsletter 7 2.
  • Clyde, M., Parmigiani, G. and Vidakovic, B. (1998). Multiple shrinkage and subset selection in wavelets. Biometrika 85 391–401.
    Mathematical Reviews (MathSciNet): MR1649120
    Zentralblatt MATH: 0938.62021
    Digital Object Identifier: doi:10.1093/biomet/85.2.391
  • Daubechies, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia.
    Mathematical Reviews (MathSciNet): MR1162107
    Zentralblatt MATH: 0776.42018
  • Donoho, D. L. and Johnstone, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 425–455.
    Mathematical Reviews (MathSciNet): MR1311089
    Zentralblatt MATH: 0815.62019
    Digital Object Identifier: doi:10.1093/biomet/81.3.425
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Statist. 24 508–539.
    Mathematical Reviews (MathSciNet): MR1394974
    Zentralblatt MATH: 0860.62032
    Digital Object Identifier: doi:10.1214/aos/1032894451
    Project Euclid: euclid.aos/1032894451
  • Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879–921.
    Mathematical Reviews (MathSciNet): MR1635414
    Zentralblatt MATH: 0935.62041
    Digital Object Identifier: doi:10.1214/aos/1024691081
    Project Euclid: euclid.aos/1024691081
  • Gradshteyn, I. S. and Ryzhik, I. M. (1980). Tables of Integrals, Series, and Products. Academic Press, New York.
    Mathematical Reviews (MathSciNet): MR0582453
    Zentralblatt MATH: 0521.33001
  • Johnstone, I. M. (2004). Function Estimation and Gaussian Sequence Models. Unpublished monograph. Available at www-stat.stanford.edu/~imj/.
  • Johnstone, I. M. and Silverman, B. W. (2004). Finding needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences. Ann. Statist. 32 1594–1649.
    Mathematical Reviews (MathSciNet): MR2089135
    Zentralblatt MATH: 1047.62008
    Digital Object Identifier: doi:10.1214/009053604000000030
    Project Euclid: euclid.aos/1091626180
  • Johnstone, I. M. and Silverman, B. W. (2004). Boundary coiflets for wavelet shrinkage in function estimation. J. Appl. Probab. 41A 81–98.
    Mathematical Reviews (MathSciNet): MR2057567
    Zentralblatt MATH: 1049.62041
    Digital Object Identifier: doi:10.1239/jap/1082552192
    Project Euclid: euclid.jap/1082552192
  • Johnstone, I. M. and Silverman, B. W. (2005). Empirical Bayes selection of wavelet thresholds. Ann. Statist. 33 1700–1752.
    Mathematical Reviews (MathSciNet): MR2166560
    Zentralblatt MATH: 1078.62005
    Digital Object Identifier: doi:10.1214/009053605000000345
    Project Euclid: euclid.aos/1123250227
  • Mallat, S. G. (1989). A theory for multiresolution signal decomposition: The wavelet representation. IEEE Trans. Pattern Analysis Machine Intelligence 11 674–693.
    Zentralblatt MATH: 0709.94650
    Digital Object Identifier: doi:10.1109/34.192463
  • Neumann, M. H. and von Sachs, R. (1995). Wavelet thresholding: Beyond the Gaussian i.i.d. situation. Wavelets and Statistics. Lecture Notes in Statist. 103 301–329. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1364677
    Zentralblatt MATH: 0831.62071
  • Shiryaev, A. N. (1996). Probability, 2nd ed. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1368405
    Zentralblatt MATH: 0909.01009
  • Simoncelli, E. P. (1997). Statistical models for images: Compression, restoration and synthesis. In Proc. 31st Asilomar Conference on Signals, Systems and Computers 673–678. IEEE Comput. Soc., Pacific Grove, CA.
  • Vidakovic, B. (1998). Nonlinear wavelet shrinkage with Bayes rules and Bayes factors. J. Amer. Statist. Assoc. 93 173–179.
    Mathematical Reviews (MathSciNet): MR1614620
    Zentralblatt MATH: 0953.62037
    Digital Object Identifier: doi:10.1080/01621459.1998.10474099
  • Vidakovic, B. (1999). Statistical Modeling by Wavelets. Wiley, New York.
    Mathematical Reviews (MathSciNet): MR1681904
    Zentralblatt MATH: 0924.62032
  • Vidakovic, B. and Ruggeri, F. (2001). BAMS method: Theory and simulations. Sankhyā Ser. B 63 234–249.
    Mathematical Reviews (MathSciNet): MR1895791

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more