The Annals of Probability

Uniform limit theorems for wavelet density estimators

Evarist Giné and Richard Nickl

Full-text: Open access

Enhanced PDF (410 KB)

Abstract

Let pn(y)=∑kα̂kϕ(y−k)+∑l=0jn−1∑kβ̂lk2l/2ψ(2ly−k) be the linear wavelet density estimator, where ϕ, ψ are a father and a mother wavelet (with compact support), α̂k, β̂lk are the empirical wavelet coefficients based on an i.i.d. sample of random variables distributed according to a density p0 on ℝ, and jn∈ℤ, jn↗∞. Several uniform limit theorems are proved: First, the almost sure rate of convergence of sup y∈ℝ|pn(y)−Epn(y)| is obtained, and a law of the logarithm for a suitably scaled version of this quantity is established. This implies that sup y∈ℝ|pn(y)−p0(y)| attains the optimal almost sure rate of convergence for estimating p0, if jn is suitably chosen. Second, a uniform central limit theorem as well as strong invariance principles for the distribution function of pn, that is, for the stochastic processes $\sqrt{n}(F_{n}^{W}(s)-F(s))=\sqrt{n}\int_{-\infty}^{s}(p_{n}-p_{0})$, s∈ℝ, are proved; and more generally, uniform central limit theorems for the processes $\sqrt{n}\int(p_{n}-p_{0})f$, $f\in\mathcal{F}$, for other Donsker classes $\mathcal{F}$ of interest are considered. As a statistical application, it is shown that essentially the same limit theorems can be obtained for the hard thresholding wavelet estimator introduced by Donoho et al. [Ann. Statist. 24 (1996) 508–539].

Article information

Source
Ann. Probab., Volume 37, Number 4 (2009), 1605-1646.

Dates
First available in Project Euclid: 21 July 2009

Permanent link to this document
https://projecteuclid.org/euclid.aop/1248182150

Digital Object Identifier
doi:10.1214/08-AOP447

Mathematical Reviews number (MathSciNet)
MR2546757

Zentralblatt MATH identifier
1255.62103

Subjects
Primary: 62G07: Density estimation
Secondary: 60F05: Central limit and other weak theorems 60F15: Strong theorems 60F17: Functional limit theorems; invariance principles

Keywords
Wavelet density estimator sup-norm loss law of the logarithm rates of convergence uniform central limit theorem wavelet thresholding adaptive estimation

Citation

Giné, Evarist; Nickl, Richard. Uniform limit theorems for wavelet density estimators. Ann. Probab. 37 (2009), no. 4, 1605--1646. doi:10.1214/08-AOP447. https://projecteuclid.org/euclid.aop/1248182150


Export citation

References

  • Bickel, P. J. and Ritov, Y. (2003). Nonparametric estimators which can be “plugged-in.” Ann. Statist. 31 1033–1053.
    Mathematical Reviews (MathSciNet): MR2001641
    Zentralblatt MATH: 1058.62031
    Digital Object Identifier: doi:10.1214/aos/1059655904
    Project Euclid: euclid.aos/1059655904
  • de la Peña, V. H. and Giné, E. (1999). Decoupling: From Dependence to Independence, Randomly Stopped Processes. U-Statistics and Processes. Martingales and Beyond. Springer, New York.
  • Daubechies, I. (1992). Ten Lectures on Wavelets. CBMS-NSF Regional Conference Series in Applied Mathematics 61. SIAM, Philadelphia, PA.
    Mathematical Reviews (MathSciNet): MR1162107
  • Deheuvels, P. (2000). Uniform limit laws for kernel density estimators on possibly unbounded intervals. In Recent Advances in Reliability Theory (Bordeaux, 2000). 477–492. Birkhäuser, Boston.
    Mathematical Reviews (MathSciNet): MR1783500
    Zentralblatt MATH: 0997.62038
  • Delyon, B. and Juditsky, A. (1996). On minimax wavelet estimators. Appl. Comput. Harmon. Anal. 3 215–228.
    Mathematical Reviews (MathSciNet): MR1400080
    Digital Object Identifier: doi:10.1006/acha.1996.0017
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1995). Wavelet shrinkage: Asymptopia? J. Roy. Statist. Soc. Ser. B 57 301–369.
    Mathematical Reviews (MathSciNet): MR1323344
    JSTOR: links.jstor.org
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Statist. 24 508–539.
    Mathematical Reviews (MathSciNet): MR1394974
    Zentralblatt MATH: 0860.62032
    Digital Object Identifier: doi:10.1214/aos/1032894451
    Project Euclid: euclid.aos/1032894451
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1997). Universal near minimaxity of wavelet shrinkage. In Festschrift for Lucien Le Cam 183–218. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1462946
    Zentralblatt MATH: 0891.62025
  • Doukhan, P. and León, J. R. (1990). Déviation quadratique d’estimateurs de densité par projections orthogonales. C. R. Acad. Sci. Paris Sér. I Math. 310 425–430.
  • Dudley, R. M. (1992). Fréchet differentiability, p-variation and uniform Donsker classes. Ann. Probab. 20 1968–1982.
    Mathematical Reviews (MathSciNet): MR1188050
    Digital Object Identifier: doi:10.1214/aop/1176989537
    Project Euclid: euclid.aop/1176989537
  • Dudley, R. M. (1999). Uniform Central Limit Theorems. Cambridge Studies in Advanced Mathematics 63. Cambridge Univ. Press, Cambridge.
    Mathematical Reviews (MathSciNet): MR1720712
  • Dvoretzky, A., Kiefer, J. and Wolfowitz, J. (1956). Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator. Ann. Math. Statist. 27 642–669.
  • Einmahl, U. and Mason, D. M. (1996). Some universal results on the behavior of increments of partial sums. Ann. Probab. 24 1388–1407.
    Mathematical Reviews (MathSciNet): MR1411499
    Zentralblatt MATH: 0872.60022
    Digital Object Identifier: doi:10.1214/aop/1065725186
    Project Euclid: euclid.aop/1065725186
  • Einmahl, U. and Mason, D. M. (2000). An empirical process approach to the uniform consistency of kernel-type function estimators. J. Theoret. Probab. 13 1–37.
    Mathematical Reviews (MathSciNet): MR1744994
    Zentralblatt MATH: 0995.62042
    Digital Object Identifier: doi:10.1023/A:1007769924157
  • Einmahl, U. and Mason, D. M. (2005). Uniform in bandwidth consistency of kernel-type function estimators. Ann. Statist. 33 1380–1403.
    Mathematical Reviews (MathSciNet): MR2195639
    Zentralblatt MATH: 1079.62040
    Digital Object Identifier: doi:10.1214/009053605000000129
    Project Euclid: euclid.aos/1120224106
  • Giné, E. and Guillou, A. (2001). On consistency of kernel density estimators for randomly censored data: Rates holding uniformly over adaptive intervals. Ann. Inst. H. Poincaré Probab. Statist. 37 503–522.
    Mathematical Reviews (MathSciNet): MR1876841
    Zentralblatt MATH: 0974.62030
    Digital Object Identifier: doi:10.1016/S0246-0203(01)01081-0
  • Giné, E. and Guillou, A. (2002). Rates of strong uniform consistency for multivariate kernel density estimators. Ann. Inst. H. Poincaré Probab. Statist. 38 907–921.
    Mathematical Reviews (MathSciNet): MR1955344
    Zentralblatt MATH: 1011.62034
    Digital Object Identifier: doi:10.1016/S0246-0203(02)01128-7
  • Giné, E., Latała, R. and Zinn, J. (2000). Exponential and moment inequalities for U-statistics. In High Dimensional Probability, II (Seattle, WA, 1999). Progress in Probability 47 (E. Giné, D. M. Mason and J. A. Wellner, eds.) 13–38. Birkhäuser, Boston.
    Mathematical Reviews (MathSciNet): MR1857343
  • Giné, E. and Nickl, R. (2008). Uniform central limit theorems for kernel density estimators. Probab. Theory Related Fields 141 333–387.
    Mathematical Reviews (MathSciNet): MR2391158
    Zentralblatt MATH: 1141.62022
    Digital Object Identifier: doi:10.1007/s00440-007-0087-9
  • Giné, E. and Nickl, R. (2009). An exponential inequality for the distribution function of the kernel density estimator, with applications to adaptive estimation. Probab. Theory Related Fields 143 569–596.
    Mathematical Reviews (MathSciNet): MR2475673
    Zentralblatt MATH: 1160.62032
    Digital Object Identifier: doi:10.1007/s00440-008-0137-y
  • Golubev, Y., Lepski, O. and Levit, B. (2001). On adaptive estimation for the sup-norm losses. Math. Methods Statist. 10 23–37.
    Mathematical Reviews (MathSciNet): MR1841807
    Zentralblatt MATH: 1005.62033
  • Hall, P., Kerkyacharian, G. and Picard, D. (1998). Block threshold rules for curve estimation using kernel and wavelet methods. Ann. Statist. 26 922–942.
    Mathematical Reviews (MathSciNet): MR1635418
    Zentralblatt MATH: 0929.62040
    Digital Object Identifier: doi:10.1214/aos/1024691082
    Project Euclid: euclid.aos/1024691082
  • Härdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. (1998). Wavelets, Approximation, and Statistical Applications. Lecture Notes in Statistics 129. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1618204
  • Juditsky, A. and Lambert-Lacroix, S. (2004). On minimax density estimation on ℝ. Bernoulli 10 187–220.
    Mathematical Reviews (MathSciNet): MR2046772
    Digital Object Identifier: doi:10.3150/bj/1082380217
    Project Euclid: euclid.bj/1082380217
  • Kerkyacharian, G. and Picard, D. (1992). Density estimation in Besov spaces. Statist. Probab. Lett. 13 15–24.
    Mathematical Reviews (MathSciNet): MR1147634
  • Kerkyacharian, G., Picard, D. and Tribouley, K. (1996). Lp adaptive density estimation. Bernoulli 2 229–247.
    Mathematical Reviews (MathSciNet): MR1416864
    Project Euclid: euclid.bj/1178291720
  • Komlós, J., Major, P. and Tusnády, G. (1975). An approximation of partial sums of independent rv’s and the sample df. I. Z. Wahrsch. Verw. Gebiete 32 111–131.
  • Ledoux, M. (2001). The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89. Amer. Math. Soc., Providence, RI.
    Mathematical Reviews (MathSciNet): MR1849347
  • Love, E. R. and Young, L. C. (1937). Sur une classe de fonctionelles linéaires. Fund. Math. 28 243–257.
  • Massiani, A. (2003). Vitesse de convergence uniforme presque sûre de l’estimateur linéaire par méthode d’ondelettes. C. R. Math. Acad. Sci. Paris 337 67–70.
  • Meyer, Y. (1992). Wavelets and Operators. Cambridge Studies in Advanced Mathematics 37. Cambridge Univ. Press, Cambridge.
    Mathematical Reviews (MathSciNet): MR1228209
  • Nickl, R. (2006). Empirical and Gaussian processes on Besov classes. In High Dimensional Probability. Institute of Mathematical Statistics Lecture Notes—Monograph Series 51 (E. Giné, V. Koltchinskii, W. Li and J. Zinn, eds.) 185–195. IMS, Beachwood, OH.
    Mathematical Reviews (MathSciNet): MR2387769
    Zentralblatt MATH: 1156.60020
    Digital Object Identifier: doi:10.1214/074921706000000842
  • Nickl, R. (2007). Donsker-type theorems for nonparametric maximum likelihood estimators. Probab. Theory Related Fields 138 411–449.
    Mathematical Reviews (MathSciNet): MR2299714
    Digital Object Identifier: doi:10.1007/s00440-006-0031-4
  • Nickl, R. and Pötscher, B. M. (2007). Bracketing metric entropy rates and empirical central limit theorems for function classes of Besov- and Sobolev-type. J. Theoret. Probab. 20 177–199.
    Mathematical Reviews (MathSciNet): MR2324525
    Zentralblatt MATH: 1130.46020
    Digital Object Identifier: doi:10.1007/s10959-007-0058-1
  • Nolan, D. and Pollard, D. (1987). U-processes: Rates of convergence. Ann. Statist. 15 780–799.
    Mathematical Reviews (MathSciNet): MR888439
    Zentralblatt MATH: 0624.60048
    Digital Object Identifier: doi:10.1214/aos/1176350374
    Project Euclid: euclid.aos/1176350374
  • Stute, W. (1982). A law of the logarithm for kernel density estimators. Ann. Probab. 10 414–422.
    Mathematical Reviews (MathSciNet): MR647513
    Zentralblatt MATH: 0493.62040
    Digital Object Identifier: doi:10.1214/aop/1176993866
    Project Euclid: euclid.aop/1176993866
  • Talagrand, M. (1994). Sharper bounds for Gaussian and empirical processes. Ann. Probab. 22 28–76.
    Mathematical Reviews (MathSciNet): MR1258865
    Zentralblatt MATH: 0798.60051
    Digital Object Identifier: doi:10.1214/aop/1176988847
    Project Euclid: euclid.aop/1176988847
  • Talagrand, M. (1996). New concentration inequalities in product spaces. Invent. Math. 126 505–563.
    Mathematical Reviews (MathSciNet): MR1419006
    Zentralblatt MATH: 0893.60001
    Digital Object Identifier: doi:10.1007/s002220050108
  • Triebel, H. (1983). Theory of Function Spaces. Birkhäuser, Basel.
    Mathematical Reviews (MathSciNet): MR781540
  • Tsybakov, A. B. (1998). Pointwise and sup-norm sharp adaptive estimation of functions on the Sobolev classes. Ann. Statist. 26 2420–2469.
    Mathematical Reviews (MathSciNet): MR1700239
    Zentralblatt MATH: 0933.62028
    Digital Object Identifier: doi:10.1214/aos/1024691478
    Project Euclid: euclid.aos/1024691478
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer Series in Statistics. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1385671
    Zentralblatt MATH: 0862.60002
  • Vidakovic, B. (1999). Statistical Modeling by Wavelets. Wiley, New York.
    Mathematical Reviews (MathSciNet): MR1681904
    Zentralblatt MATH: 0924.62032

The Institute of Mathematical Statistics

The Annals of Probability

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more