The Annals of Statistics

Wavelet regression in random design with heteroscedastic dependent errors

Rafał Kulik and Marc Raimondo

Full-text: Open access


We investigate function estimation in nonparametric regression models with random design and heteroscedastic correlated noise. Adaptive properties of warped wavelet nonlinear approximations are studied over a wide range of Besov scales, $f\in\mathcal{B}_{\pi,r}^{s}$, and for a variety of Lp error measures. We consider error distributions with Long-Range-Dependence parameter α, 0<α≤1; heteroscedasticity is modeled with a design dependent function σ. We prescribe a tuning paradigm, under which warped wavelet estimation achieves partial or full adaptivity results with the rates that are shown to be the minimax rates of convergence. For p>2, it is seen that there are three rate phases, namely the dense, sparse and long range dependence phase, depending on the relative values of s, p, π and α. Furthermore, we show that long range dependence does not come into play for shape estimation f∫f. The theory is illustrated with some numerical examples.

Article information

Ann. Statist., Volume 37, Number 6A (2009), 3396-3430.

First available in Project Euclid: 17 August 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation
Secondary: 62G08: Nonparametric regression 62G20: Asymptotic properties

Adaptive estimation nonparametric regression shape estimation random design long range dependence wavelets thresholding maxiset warped wavelets


Kulik, Rafał; Raimondo, Marc. Wavelet regression in random design with heteroscedastic dependent errors. Ann. Statist. 37 (2009), no. 6A, 3396--3430. doi:10.1214/09-AOS684.

Export citation


  • [1] Beran, J. and Feng, Y. (2001). Local polynomial estimation with a FARIMA–GARCH error process. Bernoulli 7 733–750.
  • [2] Beran, J. and Feng, Y. (2002). Local polynomial fitting with long-memory, short-memory and antipersistent errors. Ann. Inst. Statist. Math. 54 291–311.
  • [3] Cai, T. and Brown, L. D. (1999). Wavelet estimation for samples with random uniform design. Statist. Probab. Lett. 42 313–321.
  • [4] Cheng, B. and Robinson, P. M. (1991). Density estimation in strongly dependent nonlinear time series. Statist. Sinica 1 335–359.
  • [5] Cheng, B. and Robinson, P. M. (1994). Semiparametric estimation from time series with long-range dependence. J. Econometrics 64 335–353.
  • [6] Claeskens, G. and Hall, P. (2002). Effect of dependence on stochastic measures of accuracy of density estimators. Ann. Statist. 30 431–454.
  • [7] Cohen, A., Daubechies, I. and Vial, P. (1993). Wavelets on the interval and fast wavelet transforms. Appl. Comput. Harmon. Anal. 1 54–81.
  • [8] Csörgő, S. and Mielniczuk, J. (1999). Random-design regression under long-range dependent errors. Bernoulli 5 209–224.
  • [9] Csörgő, S. and Mielniczuk, J. (2000). The smoothing dichotomy in random-design regression with long-memory errors based on moving averages. Statist. Sinica 10 771–787.
  • [10] Donoho, D., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Statist. 24 508–539.
  • [11] Dzhaparidze, K. and van Zanten, J. H. (2001). On Bernstein-type inequalities for martingales. Stochastic Process. Appl. 93 109–117.
  • [12] Efromovich, S. (1999). How to overcome the curse of long-memory errors. IEEE Trans. Inform. Theory 45 1735–1741.
  • [13] Guo, H. and Koul, H. L. (2008). Asymptotic inference in some hetereoscedastic regression models with long memory design and errors. Ann. Statist. 36 458–487.
  • [14] Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Application. Academic Press, New York.
  • [15] Hall, P., Lahiri, S. N. and Truong, Y. K. (1995). On bandwidth choice for density estimation with dependent data. Ann. Statist. 23 2241–2263.
  • [16] Härdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. (1997). Wavelets, Approximation, and Statistical Applications. Lecture Notes in Statistics 129. Springer, New York.
  • [17] Johnstone, I. M. (1999). Wavelet shrinkage for correlated data and inverse problems: Adaptivity results. Statist. Sinica 9 51–83.
  • [18] Kerkyacharian, G. and Picard, D. (2000). Thresholding algorithms, maxisets and well-concentrated bases. Test 9 283–344.
  • [19] Kerkyacharian, G. and Picard, D. (2004). Regression in random design and warped wavelets. Bernoulli 10 1053–1105.
  • [20] Kulik, R. and Raimondo, M. (2009). Lp wavelet regression with correlated errors. Statist. Sinica. To appear.
  • [21] Mallat, S. (1998). A Wavelet Tour of Signal Processing, 2nd ed. Academic Press, San Diego, CA.
  • [22] Masry, E. (2001). Local linear regression estimation under long-range dependence: Strong consistency and rates. IEEE Trans. Inform. Theory 47 2863–2875.
  • [23] Masry, E. and Mielniczuk, J. (1999). Local linear regression estimation for time series with long-range dependence. Stochastic Process. Appl. 82 173–193.
  • [24] Mielniczuk, J. and Wu, W. B. (2004). On random-design model with dependent errors. Statist. Sinica 14 1105–1126.
  • [25] Pensky, M. and Sapatinas, T. (2009). Functional deconvolution in a periodic setting: Uniform case. Ann. Statist. 37 73–104.
  • [26] Robinson, P. M. and Hidalgo, F. J. (1997). Time series regression with long-range dependence. Ann. Statist. 25 77–104.
  • [27] Wang, Y. (1996). Function estimation via wavelet shrinkage for long-memory data. Ann. Statist. 24 466–484.
  • [28] Wu, W. B. and Mielniczuk, J. (2002). Kernel density estimation for linear processes. Ann. Statist. 30 1441–1459.
  • [29] Yang, Y. (2001). Nonparametric regression with dependent errors. Bernoulli 7 633–655.
  • [30] Yao, Q. and Tong, H. (1998). Cross-validatory bandwidth selections for regression estimation based on dependent data. J. Statist. Plann. Inference 68 387–415.