The Annals of Statistics

Variance estimation in nonparametric regression via the difference sequence method

Lawrence D. Brown and M. Levine

Full-text: Open access


Consider a Gaussian nonparametric regression problem having both an unknown mean function and unknown variance function. This article presents a class of difference-based kernel estimators for the variance function. Optimal convergence rates that are uniform over broad functional classes and bandwidths are fully characterized, and asymptotic normality is also established. We also show that for suitable asymptotic formulations our estimators achieve the minimax rate.

Article information

Ann. Statist., Volume 35, Number 5 (2007), 2219-2232.

First available in Project Euclid: 7 November 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression 62G20: Asymptotic properties

Nonparametric regression variance estimation asymptotic minimaxity


Brown, Lawrence D.; Levine, M. Variance estimation in nonparametric regression via the difference sequence method. Ann. Statist. 35 (2007), no. 5, 2219--2232. doi:10.1214/009053607000000145.

Export citation


  • Brown, L. D., Cai, T., Low, M. and Zhang, C.-H. (2002). Asymptotic equivalence theory for nonparametric regression with random design. Ann. Statist. 30 688–707.
  • Brown, L. D., Gans, N., Mandelbaum, A., Sakov, A., Shen, H., Zeltyn, S. and Zhao, L. (2005). Statistical analysis of a telephone call center: A queueing-science perspective. J. Amer. Statist. Assoc. 100 36–50.
  • Brown, L. D. and Low, M. (1996). A constrained risk inequality with applications to nonparametric function estimation. Ann. Statist. 24 2524–2535.
  • Buckley, M. J. and Eagleson, G. K. (1989). A graphical method for estimating the residual variance in nonparametric regression. Biometrika 76 203–210.
  • Buckley, M. J., Eagleson, G. K. and Silverman, B. W. (1988). The estimation of residual variance in nonparametric regression. Biometrika 75 189–199.
  • Carroll, R. J. (1982). Adapting for heteroscedasticity in linear models. Ann. Statist. 10 1224–1233.
  • Carter, C. K. and Eagleson, G. K. (1992). A comparison of variance estimators in nonparametric regression. J. Roy. Statist. Soc. Ser. B 54 773–780.
  • Dette, H. (2002). A consistent test for heteroscedasticity in nonparametric regression based on the kernel method. J. Statist. Plann. Inference 103 311–329.
  • Dette, H., Munk, A. and Wagner, T. (1998). Estimating the variance in nonparametric regression–-what is a reasonable choice? J. R. Stat. Soc. Ser. B Stat. Methodol. 60 751–764.
  • Diggle, P. J. and Verbyla, A. (1998). Nonparametric estimation of covariance structure in longitudinal data. Biometrics 54 401–415.
  • Donoho, D. and Liu, R. (1991). Geometrizing rates of convergence. II. Ann. Statist. 19 633–667.
  • Donoho, D. and Liu, R. (1991). Geometrizing rates of convergence. III. Ann. Statist. 19 668–701.
  • Donoho, D., Liu, R. and MacGibbon, B. (1990). Minimax risk over hyperrectangles and implications. Ann. Statist. 18 1416–1437.
  • Efromovich, S. (1996). On nonparametric regression for iid observations in a general setting. Ann. Statist. 24 1125–1144.
  • Fan, J. (1992). Design-adaptive nonparametric regression. J. Amer. Statist. Assoc. 87 998–1004.
  • Fan, J. (1993). Local linear regression smoothers and their minimax efficiencies. Ann. Statist. 21 196–216.
  • Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. Chapman and Hall, London.
  • Fan, J. and Yao, Q. (1998). Efficient estimation of conditional variance functions in stochastic regression. Biometrika 85 645–660.
  • Gasser, T., Sroka, L. and Jennen-Steinmetz, C. (1986). Residual variance and residual pattern in nonlinear regression. Biometrika 73 625–633.
  • Hall, P. and Carroll, R. (1989). Variance function estimation in regression: The effect of estimating the mean. J. Roy. Statist. Soc. Ser. B 51 3–14.
  • Hall, P., Kay, J. and Titterington, D. (1990). Asymptotically optimal difference-based estimation of variance in nonparametric regression. Biometrika 77 521–528.
  • Hall, P. and Marron, J. (1990). On variance estimation in nonparametric regression. Biometrika 77 415–419.
  • Härdle, W. and Tsybakov, A. (1997). Local polynomial estimators of the volatility function in nonparametric autoregression. J. Econometrics 81 223–242.
  • Hart, J. (1997). Nonparametric Smoothing and Lack-of-Fit Tests. Springer, New York.
  • Levine, M. (2006). Bandwidth selection for a class of difference-based variance estimators in the nonparametric regression: A possible approach. Comput. Statist. Data Anal. 50 3405–3431.
  • Matloff, N., Rose, R. and Tai, R. (1984). A comparison of two methods for estimating optimal weights in regression analysis. J. Statist. Comput. Simul. 19 265–274.
  • Müller, H.-G. and Stadtmüller, U. (1987). Estimation of heteroscedasticity in regression analysis. Ann. Statist. 15 610–625.
  • Müller, H.-G. and Stadtmüller, U. (1993). On variance function estimation with quadratic forms. J. Statist. Plann. Inference 35 213–231.
  • Munk, A., Bissantz, N., Wagner, T. and Freitag, G. (2005). On difference-based variance estimation in nonparametric regression when the covariate is high dimensional. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 19–41.
  • Munk, A. and Ruymgaart, F. (2002). Minimax rates for estimating the variance and its derivatives in nonparametric regression. Aust. N. Z. J. Statist. 44 479–488.
  • Opsomer, J., Ruppert, D., Wand, M., Holst, U. and Hössjer, O. (1999). Kriging with nonparametric variance function estimation. Biometrics 55 704–710.
  • Peligrad, M. and Utev, S. (1997). Central limit theorem for linear processes. Ann. Probab. 25 443–456.
  • Rice, J. (1984). Bandwidth choice for nonparametric kernel regression. Ann. Statist. 12 1215–1230.
  • Ruppert, D., Wand, M., Holst, U. and Hössjer, O. (1997). Local polynomial variance-function estimation. Technometrics 39 262–273.
  • Seifert, B., Gasser, T. and Wolf, A. (1993). Nonparametric estimation of the residual variance revisited. Biometrika 80 373–383.
  • Shen, H. and Brown, L. (2006). Nonparametric modelling for time-varying customer service times at a bank call center. Appl. Stoch. Models Bus. Ind. 22 297–311.
  • Shiryaev, A. N. (1996). Probability, 2nd ed. Springer, New York.
  • Spokoiny, V. (2002). Variance estimation for high-dimensional regression models. J. Multivariate Anal. 82 111–133.
  • Stone, C. J. (1980). Optimal rates of convergence for nonparametric estimators. Ann. Statist. 8 1348–1360.
  • von Neumann, J. (1941). Distribution of the ratio of the mean square successive difference to the variance. Ann. Math. Statist. 12 367–395.
  • von Neumann, J. (1942). A further remark concerning the distribution of the ratio of the mean square successive difference to the variance. Ann. Math. Statist. 13 86–88.
  • Wang, L., Brown, L., Cai, T. and Levine, M. (2006). Effect of mean on variance function estimation in nonparametric regression. Technical report, Dept. Statistics, Univ. Pennsylvania. Available at