The Annals of Statistics

General empirical Bayes wavelet methods and exactly adaptive minimax estimation

Cun-Hui Zhang

Full-text: Open access

Abstract

In many statistical problems, stochastic signals can be represented as a sequence of noisy wavelet coefficients. In this paper, we develop general empirical Bayes methods for the estimation of true signal. Our estimators approximate certain oracle separable rules and achieve adaptation to ideal risks and exact minimax risks in broad collections of classes of signals. In particular, our estimators are uniformly adaptive to the minimum risk of separable estimators and the exact minimax risks simultaneously in Besov balls of all smoothness and shape indices, and they are uniformly superefficient in convergence rates in all compact sets in Besov spaces with a finite secondary shape parameter. Furthermore, in classes nested between Besov balls of the same smoothness index, our estimators dominate threshold and James–Stein estimators within an infinitesimal fraction of the minimax risks. More general block empirical Bayes estimators are developed. Both white noise with drift and nonparametric regression are considered.

Article information

Source
Ann. Statist., Volume 33, Number 1 (2005), 54-100.

Dates
First available in Project Euclid: 8 April 2005

Permanent link to this document
https://projecteuclid.org/euclid.aos/1112967699

Digital Object Identifier
doi:10.1214/009053604000000995

Mathematical Reviews number (MathSciNet)
MR2157796

Zentralblatt MATH identifier
1064.62009

Subjects
Primary: 62C12: Empirical decision procedures; empirical Bayes procedures 62G05: Estimation 62G08: Nonparametric regression 62G20: Asymptotic properties 62C25: Compound decision problems

Keywords
Empirical Bayes wavelet adaptation minimax estimation white noise nonparametric regression threshold estimate Besov space

Citation

Zhang, Cun-Hui. General empirical Bayes wavelet methods and exactly adaptive minimax estimation. Ann. Statist. 33 (2005), no. 1, 54--100. doi:10.1214/009053604000000995. https://projecteuclid.org/euclid.aos/1112967699


Export citation

References

  • Abramovich, F., Benjamini, Y., Donoho, D. L. and Johnstone, I. M. (2000). Adapting to unknown sparsity by controlling the false discovery rate. Technical Report 2000-19, Dept. Statistics, Stanford Univ.
  • Barron, A., Birgé, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301--413.
  • Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis. Springer, New York.
  • Breiman, L., Friedman, J., Olshen, R. and Stone, C. (1984). Classification and Regression Trees. Wadsworth, Belmont, CA.
  • Brown, L. D., Cai, T. T., Low, M. and Zhang, C.-H. (2002). Asymptotic equivalence theory for nonparametric regression with random design. Ann. Statist. 30 688--707.
  • Brown, L. D. and Low, M. G. (1996). Asymptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24 2384--2398.
  • Brown, L. D., Low, M. G. and Zhao, L. H. (1997). Superefficiency in nonparametric function estimation. Ann. Statist. 25 2607--2625.
  • Cai, T. T. (1999). Adaptive wavelet estimation: A block thresholding and oracle inequality approach. Ann. Statist. 27 898--924.
  • Cai, T. T. (2000). On adaptability and information-pooling in nonparametric function estimation. Technical report, Dept. Statistics, Univ. Pennsylvania.
  • Cavalier, L. and Tsybakov, A. B. (2001). Penalized blockwise Stein's method, monotone oracles and sharp adaptive estimation. Math. Methods Statist. 10 247--282.
  • Cavalier, L. and Tsybakov, A. B. (2002). Sharp adaptation for inverse problems with random noise. Probab. Theory Related Fields 123 323--354.
  • Chui, C. K. (1992). An Introduction to Wavelets. Academic Press, San Diego, CA.
  • Daubechies, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia.
  • Donoho, D. L. and Johnstone, I. M. (1994a). Ideal spatial adaptation via wavelet shrinkage. Biometrika 81 425--455.
  • Donoho, D. L. and Johnstone, I. M. (1994b). Minimax risk over $\ell_p$-balls for $\ell_q$-error. Probab. Theory Related Fields 99 277--303.
  • Donoho, D. L. and Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200--1224.
  • Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879--921.
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1995). Wavelet shrinkage: Asymptopia? (with discussion). J. Roy. Statist. Soc. Ser. B 57 301--369.
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Statist. 24 508--539.
  • Dubuc, S. (1986). Interpolation through an iterative scheme. J. Math. Anal. Appl. 114 185--204.
  • Efromovich, S. (1985). Nonparametric estimation of a density of unknown smoothness. Theory Probab. Appl. 30 557--568.
  • Efromovich, S. (1999). Nonparametric Curve Estimation. Springer, New York.
  • Efromovich, S. and Pinsker, M. S. (1984). An adaptive algorithm for nonparametric filtering. Automat. Remote Control 11 58--65.
  • Efromovich, S. and Pinsker, M. S. (1986). An adaptive algorithm for minimax nonparametric estimation of a spectral density. Problems Inform. Transmission 22 62--76.
  • Efron, B. and Morris, C. (1973). Stein's estimation rule and its competitors---an empirical Bayes approach. J. Amer. Statist. Assoc. 68 117--130.
  • Foster, D. P. and George, E. I. (1994). The risk inflation criterion for multiple regression. Ann. Statist. 22 1947--1975.
  • Foster, D. P., Stine, R. A. and Wyner, A. J. (2002). Universal codes for finite sequences of integers drawn from a monotone distribution. IEEE Trans. Inform. Theory 48 1713--1720.
  • Friedman, J. (1991). Multivariate adaptive regression splines (with discussion). Ann. Statist. 19 1--141.
  • Friedman, J. and Silverman, B. W. (1989). Flexible parsimonious smoothing and additive modeling. Technometrics 31 3--21.
  • Golubev, G. K. (1992). Nonparametric estimation of smooth probability densities in $L_2$. Problems Inform. Transmission 28 44--54.
  • Grama, I. and Nussbaum, M. (1998). Asymptotic equivalence for nonparametric generalized linear models. Probab. Theory Related Fields 111 167--214.
  • Hall, P., Kerkyacharian, G. and Picard, D. (1998). Block threshold rules for curve estimation using kernel and wavelet methods. Ann. Statist. 26 922--942.
  • Hall, P., Kerkyacharian, G. and Picard, D. (1999). On the minimax optimality of block thresholding wavelet estimators. Statist. Sinica 9 33--49.
  • Hall, P. and Patil, P. (1995). Formulae for mean integrated squared error of nonlinear wavelet-based density estimators. Ann. Statist. 23 905--928.
  • Hall, P. and Patil, P. (1996). Effect of threshold rules on performance of wavelet-based curve estimators. Statist. Sinica 6 331--345.
  • Härdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. (1998). Wavelets, Approximation, and Statistical Applications. Lecture Notes in Statist. 129. Springer, New York.
  • Hastie, T., Tibshirani, R. and Friedman, J. H. (2001). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York.
  • Ibragimov, I. A. and Khas'minskii, R. Z. (1981). Statistical Estimation: Asymptotic Theory. Springer, New York.
  • James, W. and Stein, C. (1961). Estimation with quadratic loss. Proc. Fourth Berkeley Symp. Math. Statist. Probab. 1 361--379. Univ. California Press, Berkeley.
  • Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1992). Estimation d'une densité de probabilité par méthode d'ondelettes. C. R. Acad. Sci. Paris Sér. I Math. 315 211--216.
  • Johnstone, I. M. and Silverman, B. W. (1998). Empirical Bayes approaches to mixture problems and wavelet regression. Technical report, Dept. Statistics, Stanford Univ. Available at http://www-stat.stanford.edu/people/faculty/johnstone/techreports.html.
  • Johnstone, I. M. and Silverman, B. W. (2004). Needles and hay in haystacks: Empirical Bayes estimates of possibly sparse sequences. Ann. Statist. 32 1594--1649.
  • Johnstone, I. M. and Silverman, B. W. (2005). Empirical Bayes selection of wavelet thresholds. Ann. Statist. 33. To appear.
  • Juditsky, A. (1997). Wavelet estimators: Adapting to unknown smoothness. Math. Methods Statist. 6 1--25.
  • Lepski, O. V., Mammen, E. and Spokoiny, V. G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors. Ann. Statist. 25 929--947.
  • Morris, C. N. (1983). Parametric empirical Bayes inference: Theory and applications (with discussion). J. Amer. Statist. Assoc. 78 47--65.
  • Nussbaum, M. (1996). Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Statist. 24 2399--2430.
  • Robbins, H. (1951). Asymptotically subminimax solutions of compound statistical decision problems. Proc. Second Berkeley Symp. Math. Statist. Probab. 131--148. Univ. California Press, Berkeley.
  • Robbins, H. (1956). An empirical Bayes approach to statistics. Proc. Third Berkeley Symp. Math. Statist. Probab. 1 157--163. Univ. California Press, Berkeley.
  • Robbins, H. (1983). Some thoughts on empirical Bayes estimation. Ann. Statist. 11 713--723.
  • Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proc. Third Berkeley Symp. Math. Statist. Probab. 1 197--206. Univ. California Press, Berkeley.
  • Stein, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135--1151.
  • Stone, C. J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation (with discussion). Ann. Statist. 22 118--184.
  • Zhang, C.-H. (1990). Fourier methods for estimating mixing densities and distributions. Ann. Statist. 18 806--831.
  • Zhang, C.-H. (1997). Empirical Bayes and compound estimation of normal means. Statist. Sinica 7 181--193.
  • Zhang, C.-H. (2000). General empirical Bayes wavelet methods. Technical Report 2000-007, Dept. Statistics, Rutgers Univ. Available at http://stat.rutgers.edu/~cunhui/papers.