The Annals of Statistics

Bootstrap confidence bands for regression curves and their derivatives

Gerda Claeskens and Ingrid van Keilegom

Full-text: Open access


Confidence bands for regression curves and their first p derivatives are obtained via local pth order polynomial estimation. The method allows for multiparameter local likelihood estimation as well as other unbiased estimating equations. As an alternative to the confidence bands obtained by asymptotic distribution theory, we also study smoothed bootstrap confidence bands. Simulations illustrate the finite sample properties of the methodology.

Article information

Ann. Statist., Volume 31, Number 6 (2003), 1852-1884.

First available in Project Euclid: 16 January 2004

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G15: Tolerance and confidence regions
Secondary: 62E20: Asymptotic distribution theory 62G09: Resampling methods

Confidence band lack-of-fit test local estimation equations local polynomial esimation multiparameter local likelihood one-step bootstrap smoothed bootstrap


Claeskens, Gerda; Keilegom, Ingrid van. Bootstrap confidence bands for regression curves and their derivatives. Ann. Statist. 31 (2003), no. 6, 1852--1884. doi:10.1214/aos/1074290329.

Export citation


  • Aerts, M. and Claeskens, G. (1997). Local polynomial estimation in multiparameter likelihood models. J. Amer. Statist. Assoc. 92 1536--1545.
  • Aerts, M. and Claeskens, G. (2001). Bootstrap tests for misspecified models, with application to clustered binary data. Comput. Statist. Data Anal. 36 383--401.
  • Arnold, B. C., Castillo, E. and Sarabia, J.-M. (1992). Conditionally Specified Distributions. Lecture Notes in Statist. 73. Springer, New York.
  • Arnold, B. C. and Strauss, D. (1991). Pseudolikelihood estimation: Some examples. Sankhyā Ser. B 53 233--243.
  • Beirlant, J. and Goegebeur, Y. (2004). Local polynomial maximum likelihood estimation for Pareto-type distributions. J. Multivariate Anal. To appear.
  • Bickel, P. J. and Rosenblatt, M. (1973). On some global measures of the deviations of density function estimates. Ann. Statist. 1 1071--1095.
  • Carroll, R. J., Ruppert, D. and Welsh, A. H. (1998). Local estimating equations. J. Amer. Statist. Assoc. 93 214--227.
  • Claeskens, G. and Aerts, M. (2000). Bootstrapping local polynomial estimators in likelihood-based models. J. Statist. Plann. Inference 86 63--80.
  • Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. J. Amer. Statist. Assoc. 74 829--836.
  • Davison, A. C. and Ramesh, N. I. (2000). Local likelihood smoothing of sample extremes. J. R. Stat. Soc. Ser. B Stat. Methodol. 62 191--208.
  • Einmahl, J. H. J., Ruymgaart, F. H. and Wellner, J. A. (1988). A characterization of weak convergence of weighted multivariate empirical processes. Acta Sci. Math. (Szeged) 52 191--205.
  • Eubank, R. L. and Speckman, P. L. (1993). Confidence bands in nonparametric regression. J. Amer. Statist. Assoc. 88 1287--1301.
  • Fan, J. (1992). Design-adaptive nonparametric regression. J. Amer. Statist. Assoc. 87 998--1004.
  • Fan, J. (1993). Local linear regression smoothers and their minimax efficiencies. Ann. Statist. 21 196--216.
  • Fan, J. and Chen, J. (1999). One-step local quasi-likelihood estimation. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 927--943.
  • Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. Chapman and Hall, London.
  • Fan, J., Heckman, N. E. and Wand, M. P. (1995). Local polynomial kernel regression for generalized linear models and quasi-likelihood functions. J. Amer. Statist. Assoc. 90 141--150.
  • Fan, J. and Zhang, W. (2000). Simultaneous confidence bands and hypothesis testing in varying-coefficient models. Scand. J. Statist. 27 715--731.
  • Hall, P. (1979). On the rate of convergence of normal extremes. J. Appl. Probab. 16 433--439.
  • Hall, P. (1991a). Edgeworth expansions for nonparametric density estimators, with applications. Statistics 22 215--232.
  • Hall, P. (1991b). On convergence rates of suprema. Probab. Theory Related Fields 89 447--455.
  • Hall, P. (1992). Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density. Ann. Statist. 20 675--694.
  • Hall, P. and Titterington, D. M. (1988). On confidence bands in nonparametric density estimation and regression. J. Multivariate Anal. 27 228--254.
  • Härdle, W. (1989). Asymptotic maximal deviation of $M$-smoothers. J. Multivariate Anal. 29 163--179.
  • Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. Proc. Fifth Berkeley Symp. Math. Statist. Probab. 1 221--233. Univ. California Press, Berkeley.
  • Johnston, G. J. (1982). Probabilities of maximal deviations for nonparametric regression function estimates. J. Multivariate Anal. 12 402--414.
  • Knafl, G., Sacks, J. and Ylvisaker, D. (1985). Confidence bands for regression functions. J. Amer. Statist. Assoc. 80 683--691.
  • McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd ed. Chapman and Hall, London.
  • Neumann, N. H. and Polzehl, J. (1998). Simultaneous bootstrap confidence bands in nonparametric regression. J. Nonparametr. Statist. 9 307--333.
  • Rosenblatt, M. (1952). Remarks on a multivariate transformation. Ann. Math. Statist. 23 470--472.
  • Ruppert, D. and Wand, M. P. (1994). Multivariate locally weighted least squares regression. Ann. Statist. 22 1346--1370.
  • Ruppert, D., Wand, M. P., Holst, U. and Hössjer, O. (1997). Local polynomial variance function estimation. Technometrics 39 262--273.
  • Shorack, G. R. and Wellner, J. A. (1986). Empirical Processes with Applications to Statistics. Wiley, New York.
  • Silverman, B. W. and Young, G. A. (1987). The bootstrap: To smooth or not to smooth? Biometrika 74 469--479.
  • Staniswalis, J. G. (1989). The kernel estimate of a regression function in likelihood-based models. J. Amer. Statist. Assoc. 84 276--283.
  • Sun, J. and Loader, C. R. (1994). Simultaneous confidence bands for linear regression and smoothing. Ann. Statist. 22 1328--1345.
  • Tusnády, G. (1977). A remark on the approximation of the sample df in the multidimensional case. Period. Math. Hungar. 8 53--55.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York.
  • Wedderburn, R. W. M. (1974). Quasi-likelihood functions, generalized linear models, and the Gauss--Newton method. Biometrika 61 439--447.
  • Xia, Y. (1998). Bias-corrected confidence bands in nonparametric regression. J. R. Stat. Soc. Ser. B Stat. Methodol. 60 797--811.
  • Zhao, P.-L. (1994). Asymptotics of kernel estimators based on local maximum likelihood. J. Nonparametr. Statist. 4 79--90.