• Bernoulli
  • Volume 7, Number 4 (2001), 633-655.

Nonparametric regression with dependent errors

Yuhong Yang

Full-text: Open access


We study minimax rates of convergence for nonparametric regression under a random design with dependent errors. It is shown that when the errors are independent of the explanatory variables, long-range dependence among the errors does not necessarily hurt regression estimation, which at first glance contradicts with earlier results by Hall and Hart, Wang, and Johnstone and Silverman under a fixed design. In fact we show that, in general, the minimax rate of convergence under the square $L_2$ loss is simply at the worse of two quantities: one determined by the massiveness of the class alone and the other by the severity of the dependence among the errors alone. The clear separation of the effects of the function class and dependence among the errors in determining the minimax rate of convergence is somewhat surprising. Examples of function classes under different covariance structures including both short- and long-range dependences are given.

Article information

Bernoulli, Volume 7, Number 4 (2001), 633-655.

First available in Project Euclid: 17 March 2004

Permanent link to this document

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

long-range dependent errors minimax rate of convergence nonparametric regression


Yang, Yuhong. Nonparametric regression with dependent errors. Bernoulli 7 (2001), no. 4, 633--655.

Export citation


  • [1] Adenstedt, R.K. (1974) On large sample estimation for the mean of a stationary sequence. Ann. Statist., 2, 1095-1107.
  • [2] Barron, A.R. (1987) Are Bayes rules consistent in information? In T.M. Cover and B. Gopinath (eds), Open Problems in Communication and Computation, pp. 85-91. Berlin: Springer-Verlag.
  • [3] Beran, J. (1986) Estimation, testing and prediction for self-similar and related processes. Doctoral thesis, ETH, Zurich.
  • [4] Beran, J. (1994) Statistics for Long-Memory Processes. London: Chapman & Hall.
  • [5] Birgé, L. (1983) Approximation dans les espaces métriques et théorie de l'estimation. Z. Wahrscheinlichkeitstheorie Verw. Geb., 65, 181-237.
  • [6] Birgé, L. (1986) On estimating a density using Hellinger distance and some other strange facts. Probab. Theory Related Fields, 71, 271-291.
  • [7] Bretagnolle, J. and Huber, C. (1979) Estimation des densités: risque minimax. Z. Wahrscheinlichkeitstheorie Verw. Geb., 47, 119-137.
  • [8] Cox, D.R. (1984) Long-range dependence: a review. In H.A. David and H.T. David (eds), Statistics: An Appraisal. Proceedings of a Conference Marking the 50th Anniversary of the Statistical Laboratory, Iowa State University, pp. 55-74. Ames: Iowa State University Press.
  • [9] Dahlhaus, R. (1989) Efficient parameter estimation for self-similar processes. Ann. Statist., 17, 1749- 1766. Abstract can also be found in the ISI/STMA publication
  • [10] DeVore, R.A. and Lorentz, G.G. (1993) Constructive Approximation. Berlin: Springer-Verlag.
  • [11] Efromovich, S. (1999) How to overcome the curse of long-memory errors. IEEE Trans. Inform. Theory, 45, 1735-1741.
  • [12] Fox, R. and Taqqu, M.S. (1985) Non-central limit theorems for quadratic forms in random variables having long-range dependence. Ann. Probab., 13, 428-446.
  • [13] Giraitis, L. and Surgailis, D. (1990) A central limit theorem for quadratic forms in strongly dependent linear variables and application to asymptotical normality of Whittle's estimate. Probab. Theory Related Fields, 86, 87-104.
  • [14] Granger, C.W.J. and Joyeux, R. (1980) An introduction to long-range time series models and fractional differencing. J. Time Ser. Anal., 1, 15-30.
  • [15] Grenander, U. and Szegö, G. (1958) Toeplitz Forms and Their Applications. Berkeley: University of California Press.
  • [16] Hall, P. and Hart, J.D. (1990a) Nonparametric regression with long-range dependence. Stochastic Process. Appl., 36, 339-351. Abstract can also be found in the ISI/STMA publication
  • [17] Hall, P. and Hart, J.D. (1990b) Convergence rates in density estimation for data from infinite-order moving average processes. Probab. Theory Related Fields, 87, 253-274. Abstract can also be found in the ISI/STMA publication
  • [18] Hosking, J.R.M. (1981) Fractional differencing. Biometrika, 68, 165-176.
  • [19] Ibragimov, I.A. and Hasminskii, R.Z. (1977) On the estimation of an infinite-dimensional parameter in Gaussian white noise. Soviet Math. Dokl., 18, 1307-1309.
  • [20] Johnstone, I.M. and Silverman, B.W. (1997) Wavelet threshold estimators for data with correlated noise. J. Roy. Statist. Soc. Ser. B, 59, 319-351. Abstract can also be found in the ISI/STMA publication
  • [21] Kolmogorov, A.N. and Tihomirov, V.M. (1959) E-entropy and E-capacity of sets in functional spaces. Uspekhi Mat. Nauk, 14, 3..86. English translation 1961: S.N. Cernikov et al., Twelve Papers on Algebra and Real Functions, Amer. Math. Soc. Transl. (2), 17, pp. 277-364. Providence, RI: American Mathematical Society.
  • [22] Künsch, H., Beran, J. and Hampel, F. (1993) Contrasts under long-range correlations. Ann. Statist., 21, 943-964.
  • [23] Le Cam, L.M. (1975) On local and global properties in the theory of asymptotic normality of experiments. In M. Puri (ed.), Stochastic Processes and Related Topics, Vol. 1, pp. 13-54. New York: Academic Press.
  • [24] Le Cam, L.M. (1986) Asymptotic Methods in Statistical Decision Theory. Berlin: Springer-Verlag.
  • [25] Lorentz, G.G., Golitschek, M. v. and Makovoz, Y. (1996) Constructive Approximation: Advanced Problems. Berlin: Springer-Verlag.
  • [26] Mandelbrot, B.B. and Van Ness, J.W. (1968) Fractional Brownian motions, fractional noises and applications. SIAM Rev., 10, 422-437.
  • [27] Robinson, P.M. (1995) Gaussian semiparametric estimation of long-range dependence. Ann. Statist., 23, 1630-1661. Abstract can also be found in the ISI/STMA publication
  • [28] Robinson, P.M. (1997) Large-sample inference for nonparametric regression with dependent errors. Ann. Statist., 25, 2054-2083. Abstract can also be found in the ISI/STMA publication
  • [29] Samarov, A. and Taqqu, M.S. (1988) On the efficiency of the sample mean in long-memory noise. J. Time Ser. Anal., 9, 191-200. Abstract can also be found in the ISI/STMA publication
  • [30] Triebel, H. (1975) Interpolation properties of E-entropy and diameters. Geometric characteristics of embedding for function spaces of Sobolev-Besov type. Mat. Sb., 98, 27-41. English translation
  • [31] (1977): Math. USSR Sb., 27, 23-37.
  • [32] Wang, Y. (1996) Function estimation via wavelet shrinkage for long-memory data. Ann. Statist., 24, 466-484. Abstract can also be found in the ISI/STMA publication
  • [33] Yang, Y. and Barron, A.R. (1999) Information-theoretic determination of minimax rates of convergence. Ann. Statist., 27, 1564-1599. Abstract can also be found in the ISI/STMA publication
  • [34] Yajima, Y. (1991) Asymptotic properties of LSE in a regression model with long-memory stationary errors. Ann. Statist., 19, 158-177. Abstract can also be found in the ISI/STMA publication
  • [35] Yatracos, Y.G. (1988) A lower bound on the error in nonparametric regression type problems. Ann. Statist., 16, 1180-1187. Abstract can also be found in the ISI/STMA publication
  • [36] Yu, B. (1997) Assouad, Fano, and Le Cam. In D. Pollard, E. Torgersen and G.L. Yang (eds), Research Papers in Probability and Statistics: Festschrift in Honor of Lucien Le Cam. New York: Springer- Verlag.