The Annals of Statistics

Regression-type inference in nonparametric autoregression

Jens-Peter Kreiss and Michael H. Neumann

Full-text: Open access


We derive a strong approximation of a local polynomial estimator LPE in nonparametric autoregression by an (LPE) in a corresponding nonparametric regression model. This generally suggests the application of regression-typical tools for statistical inference in nonparametric autoregressive models. It provides an important simplification for the boot-strap method to be used: It is enough to mimic the structure of a nonparametric regression model rather than to imitate the more complicated process structure in the autoregressive case. As an example we consider a simple wild bootstrap, which is used for the construction of simultaneous confidence bands and nonparametric supremum-type tests.

Article information

Ann. Statist., Volume 26, Number 4 (1998), 1570-1613.

First available in Project Euclid: 21 June 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation 62M05: Markov processes: estimation
Secondary: 62G09: Resampling methods 62G15: Tolerance and confidence regions

Nonparametric autoregression nonparametric regression strong approximation bootstrap wild bootstrap confidence bands


Neumann, Michael H.; Kreiss, Jens-Peter. Regression-type inference in nonparametric autoregression. Ann. Statist. 26 (1998), no. 4, 1570--1613. doi:10.1214/aos/1024691254.

Export citation


  • AMOSOVA, N. N. 1972. On limit theorems for probabilities of moderate deviations. Vestnik Z. Leningrad. Univ. 13 5 14 in Russian. Z.
  • BROWN, L. D. and LOW, M. G. 1996. Asy mptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24 2384 2398. Z.
  • CSORGO, M. and REVESZ, P. 1981. Strong Approximations in Probability and Statistics. Aca¨ ´ ´ demic, New York. Z.
  • DAVy DOV, YU. A. 1970. The invariance principle for stationary processes. Theory Probab. Appl. 15 487 498. Z.
  • DOUKHAN, P. 1994. Mixing: Properties and Examples. Lecture Notes in Statist. 85. Springer, New York. Z.
  • DOUKHAN, P., MASSART, P. and RIO, E. 1995. Invariance principles for absolutely regular empirical processes. Ann. Inst. H. Poincare Probab. Statist. 31 393 427. ´ Z.
  • FAN, J. 1992. Design-adaptive nonparametric regression. J. Amer. Statist. Assoc. 87 998 1004. Z.
  • FAN, J. 1993. Local linear regression smoothers and their minimax efficiencies. Ann. Statist. 21 196 216. Z.
  • FAN, J. and GIJBELS, I. 1992. Variable bandwidth and local linear regression smoothers. Ann. Statist. 20 2008 2036. Z.
  • FAN, J. and GIJBELS, I. 1995. Local Poly nomial Modeling and Its Application Theory and Methodologies. Chapman and Hall, New York. Z.
  • FRANKE, J., KREISS, J.-P. and MAMMEN, E. 1997. Bootstrap of kernel smoothing in nonlinear time series. Discussion Paper 20 97, SFB 373, Humboldt Univ., Berlin. Z.
  • FRANKE, J., KREISS, J.-P., MAMMEN, E. and NEUMANN, M. H. 1998. Properties of the nonparametric autoregressive bootstrap. Unpublished manuscript.
  • HALL, P. 1991. On the distribution of suprema. Probab. Theory Related Fields 89 447 455. Z.
  • HALL, P. and HEy DE, C. C. 1980. Martingale Limit Theory and Its Application. Academic Press, New York. Z.
  • HALL, P. and TITTERINGTON, M. 1988. On confidence bands in nonparametric density estimation and regression. J. Multvariate Anal. 27 228 254. Z.
  • HARDLE, W. and MAMMEN, E. 1993. Comparing nonparametric versus parametric regression ¨ fits. Ann. Statist. 21 1926 1947. Z.
  • HARDLE, W. and TSy BAKOV, A. B. 1997. Local poly nomial estimators of the volatility function in ¨ nonparametric autoregression. J. Econometrics 81 223 242. Z.
  • HART, J. D. 1995. Some automated methods of smoothing time-dependent data. J. Nonparametr. Statist. 6 115 142. Z.
  • JONES, D. A. 1978. Nonlinear regressive processes. Proc. Roy. Soc. London Ser. A 360 71 95. Z.
  • KNAFL, G., SACKS, J. and YLVISAKER, D. 1985. Confidence bands for regression functions. J. Amer. Statist. Assoc. 80 683 691. Z.
  • KOMLOS, J., MAJOR, P. and TUSNADY, G. 1975. An approximation of partial sums of independent ´ ´ rv's and the sample distribution function. Z. Wahrsch. Verw. Gebiete 32 111 131. Z.
  • KONAKOV, V., LAUTER, H. and LIERO, H. 1995. Comparison of the asy mptotic power of tests ¨ based on L and L -norms under non-standard local alternatives. Discussion Paper 2 10 95, SFB 373, Humboldt Univ., Berlin. Z.
  • KOROSTELEV, A. P. and TSy BAKOV, A. B. 1993. Minimax Theory of Image Reconstruction. Lecture Notes in Statist. 82. Springer, New York. Z.
  • KREISS, J.-P., NEUMANN, M. H. and YAO, Q. 1998. Bootstrap tests for simple structures in nonparametric time series regression. Preprint No. 98 07, TU Braunschweig. Z.
  • MAMMEN, E. 1992. When Does Bootstrap Work? Asy mptotic Results and Simulations. Lecture Notes in Statist. 77. Springer, New York. Z.
  • MASRY, E. 1996. Multivariate local poly nomial regression for time series: uniform strong consistency and rates. J. Time Ser. Anal. 17 571 599. Z.
  • MASRY, E. and TJøSTHEIM, D. 1995. Nonparametric estimation and identification of nonlinear ARCH time series. Econometric Theory 11 258 289. Z.
  • NEUMANN, M. H. 1995. Automatic bandwidth choice and confidence intervals in nonparametric regression. Ann. Statist. 23 1937 1959. Z.
  • NEUMANN, M. H. 1997. On robustness of model-based bootstrap schemes in nonparametric time series analysis. Discussion Paper 88 97, SFB 373, Humboldt Univ., Berlin. Z.
  • NEUMANN, M. H. 1998. Strong approximation of density estimators from weakly dependent observations by density estimators from independent observations. Ann. Statist. 26 To appear. Z.
  • NEUMANN, M. H. and KREISS, J.-P. 1996. Bootstrap confidence bands for the autoregression function. Preprint 263, Weierstrass Institute, Berlin. Z.
  • NEUMANN, M. H. and POLZEHL, J. 1995. Simultaneous bootstrap confidence bands in nonparametric regression. J. Nonparametr. Statist. To appear. Z.
  • NUSSBAUM, M. 1996. Asy mptotic equivalence of density estimation and Gaussian white noise. Ann. Statist. 24 2399 2430. Z.
  • ROBINSON, P. M. 1983. Nonparametric estimators for time series. J. Time Ser. Anal. 4 185 207.Z.
  • SAKHANENKO, A. I. 1991. On the accuracy of normal approximation in the invariance principle. Siberian Adv. Math. 1 58 91. Z.
  • SHORACK, G. R. and WELLNER, J. A. 1986. Empirical Processes with Applications to Statistics. Wiley, New York. Z.
  • SKOROKHOD, A. V. 1965. Studies in the Theory of Random Processes. Addison-Wesley, Reading, MA. Z.
  • SPOKOINY, V. G. 1996. Adaptive and spatially adaptive testing of a nonparametric hy pothesis. Math. Methods Statist. To appear. Z.
  • STONE, C. J. 1977. Consistent nonparametric regression. Ann. Statist. 5 595 620. Z.
  • SUN, J. and LOADER, C. R. 1994. Simultaneous confidence bands for linear regression and smoothing. Ann. Statist. 22 1328 1345.
  • TSy BAKOV, A. B. 1986. Robust reconstruction of functions by the local approximation method. Problems Inform. Transmission 22 133 146. Z.
  • WU, C. F. J. 1986. Jackknife, bootstrap and other resampling methods in regression analysis. Ann. Statist. 14 1261 1343.