The Annals of Statistics

Strong approximation of density estimators from weakly dependent observations by density estimators from independent observations

Michael H. Neumann

Full-text: Open access


We derive an approximation of a density estimator based on weakly dependent random vectors by a density estimator built from independent random vectors. We construct, on a sufficiently rich probability space, such a pairing of the random variables of both experiments that the set of observations $X_1,\ldots,X_n}$ from the time series model is nearly the same as the set of observations $Y_1,\ldots,Y_n}$ from the i.i.d. model. With a high probability, all sets of the form $({X_1,\ldots,X_n}\\Delta{Y_1,\ldots,Y_n})\bigcap([a_1,b_1]\times\ldots\times[a_d,b_d])$ contain no more than $O({[n^1/2 \prod(b_i-a_i)]+ 1} \log(n))$ elements, respectively. Although this does not imply very much for parametric problems, it has important implications in nonparametric statistics. It yields a strong approximation of a kernel estimator of the stationary density by a kernel density estimator in the i.i.d. model. Moreover, it is shown that such a strong approximation is also valid for the standard bootstrap and the smoothed bootstrap. Using these results we derive simultaneous confidence bands as well as supremum­type nonparametric tests based on reasoning for the i.i.d. model.

Article information

Ann. Statist., Volume 26, Number 5 (1998), 2014-2048.

First available in Project Euclid: 21 June 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation
Secondary: 62G09: Resampling methods 62M07: Non-Markovian processes: hypothesis testing

Density estimation strong approximation bootstrap weak dependence mixing whitening by windowing simultaneous confidence bands nonparametric tests


Neumann, Michael H. Strong approximation of density estimators from weakly dependent observations by density estimators from independent observations. Ann. Statist. 26 (1998), no. 5, 2014--2048. doi:10.1214/aos/1024691367.

Export citation


  • [1] Amosova, N. N. (1972). On limit theorems for probabilities of moderate deviations. Vestnik Leningrad. Univ. 13 5-14 (in Russian).
  • [2] Ango Nze, P. (1992). Crit eres d'ergodicit´e de quelques mod eles a repr´esentation markovienne. C. R. Acad. Sci. Paris S er. I Math. 315 1301-1304.
  • [3] Ango Nze, P. and Doukhan, P. (1993). Estimation fonctionnelle de s´eries temporelles m´elangeantes. C. R. Acad. Sci. Paris S er. I Math. 317 405-408.
  • [4] Ango Nze, P. and Rios, R. (1995). Estimation L de la fonction de densit´e d`un processus faiblement d´ependant: les cas absolument r´egulier et fortement m´elangeant. C. R. Acad. Sci. Paris S er. I Math. 320 1259-1262.
  • [5] Bickel, P. and Rosenblatt, M. (1973). On some global measures of the deviations of density function estimates. Ann. Statist. 1 1071-1095.
  • [6] Br´emaud, P. and Massouli´e, L. (1996). Stability of nonlinear Hawkes processes. Ann. Probab. 24 1563-1588.
  • [7] B ¨uhlmann, P. (1994). Blockwise bootstrapped empirical process for stationary sequences. Ann. Statist. 22 995-1012.
  • [8] Dhompongsa, S. (1984). A note on the almost sure approximation of the empirical process of weakly dependent random vectors. Yokohama Math. J. 32 113-121.
  • [9] Doukhan, P. (1994). Mixing: Properties and Examples. Lecture Notes in Statist. 85. Springer, New York.
  • [10] Doukhan, P., Massart, P. and Rio, E. (1995). Invariance principles for absolutely regular empirical processes. Ann. Inst. H. Poincar´e Probab. Statist. 31 393-427.
  • [11] Doukhan, P. and Portal, F. (1987). Principe d'invariance faible pour la fonction de r´epartition empirique dans un cadre multidimensionnel et m´elangeant. Probab. Math. Statist. 8 117-132.
  • [12] Efron, B. (1979). Bootstrap methods: another look at the jackknife. Ann. Statist. 7 1-26.
  • [13] Efron, B. (1982). The Jackknife, the Bootstrap and Other Resampling Plans. SIAM, Philadelphia.
  • [14] Falk, M. and Reiss, R.-D. (1989a). Weak convergence of smoothed and nonsmoothed bootstrap quantile estimates. Ann. Probab. 17 362-371.
  • [15] Falk, M. and Reiss, R.-D. (1989b). Bootstrapping the distance between smooth bootstrap and sample quantile distribution. Probab. Theory Related Fields 82 177-186.
  • [16] Faraway, J. J. and Jhun, M. (1990). Bootstrap choice of bandwidth for density estimation. J. Amer. Statist. Assoc. 85 1119-1122.
  • [17] Franke, J., Kreiss, J.-P., Mammen, E. and Neumann, M. H. (1998). Properties of the nonparametric autoregressive bootstrap. Discussion Paper 54/98, SFB 373, Humboldt Univ., Berlin.
  • [18] Friedman, J., Stuetzle, W. and Schroeder, A. (1984). Projection pursuit density estimation. J. Amer. Statist. Assoc. 79 599-608.
  • [19] Gy ¨orfi, L., H¨ardle, W., Sarda, P. and Vieu, P. (1989). Nonparametric Curve Estimation from Time Series. Springer, Berlin.
  • [20] Hall, P. (1991). On convergence rates of suprema. Probab. Theory Related Fields 89 447- 455.
  • [21] Hall, P. (1992). The Bootstrap and Edgeworth Expansion. Springer, New York.
  • [22] Hall, P. (1993). On Edgeworth expansion and bootstrap confidence bands in nonparametric curve estimation. J. R. Statist. Soc. Ser. B 55 291-304.
  • [23] Hall, P., DiCiccio, T. J. and Romano, J. P. (1989). On smoothing and the bootstrap. Ann. Statist. 17 692-704.
  • [24] Hall, P. and Hart, J. D. (1990). Convergence rates in density estimation for data from infinite-order moving average processes. Probab. Theory Related Fields 87 253-274.
  • [25] H¨ardle, W. and Mammen, E. (1993). Comparing nonparametric versus parametric regression fits. Ann. Statist. 21 1926-1947.
  • [26] Hart, J. D. (1996). Some automated methods of smoothing time-dependent data. J. Nonparametr. Statist. 6 115-142.
  • [27] Koml ´os, J., Major, P. and Tusn´ady, G. (1975). An approximation of partial sums of independent rv's and the sample distribution function. Z. Wahrsch. Verw. Gebiete 32 111-131.
  • [28] Konakov, V., L¨auter, H. and Liero, H. (1995). Nonparametric versus parametric goodness of fit. Statistics. To appear.
  • [29] K ¨unsch, H. R. (1989). The jackknife and the bootstrap for general stationary observations. Ann. Statist. 17 1217-1241.
  • [30] Masry, E. (1994). Probability density estimation from dependent observations using wavelet orthonormal bases. Statist. Probab. Lett. 21 181-194.
  • [31] Mokkadem, A. (1988). Mixing properties of ARMA processes. Stochastic Process. Appl. 29 309-315.
  • [32] Mokkadem, A. (1990). Propri´et´es de m´elange des processus autor´egessifs poly nomiaux. Ann. Inst. H. Poincar´e Probab. Statist. 26 219-260.
  • [33] Neumann, M. H. (1995). Automatic bandwidth choice and confidence intervals in nonparametric regression. Ann. Statist. 23 1937-1959.
  • [34] Neumann, M. H. and Kreiss, J.-P. (1998). Regression-ty pe inference in nonparametric autoregression. Ann. Statist. 26 1570-1613.
  • [35] Neumann, M. H. and Paparoditis, E. (1998). A nonparametric test for the stationary density. Discussion Paper 58/98, SFB 373, Humboldt Univ., Berlin.
  • [36] Pham, T. D. (1986). The mixing property of bilinear and generalised random coefficient autoregressive models. Stochastic Process. Appl. 23 291-300.
  • [37] Pham, T. D. and Tran, L. T. (1985). Some mixing properties of time series models. Stochastic Process. Appl. 19 297-303.
  • [38] Reiss, R.-D. (1993). A Course on Point Processes. Springer, New York.
  • [39] Robinson, P. M. (1983). Nonparametric estimators for time series. J. Time Ser. Anal. 4 185-207.
  • [40] Shorack, G. R. and Wellner, J. A. (1986). Empirical Processes with Applications to Statistics. Wiley, New York.
  • [41] Silverman, B. W. and Young, G. A. (1987). The bootstrap: to smooth or not to smooth? Biometrika 74 469-479.
  • [42] Spokoiny, V. G. (1996). Adaptive and spatially adaptive testing of a nonparametric hy pothesis. Math. Methods Statist. To appear.
  • [43] Takahata, H. and Yoshihara, K. (1987). Central limit theorems for integrated square error of nonparametric density estimators based on absolutely regular random sequences. Yokohama Math. J. 35 95-111.
  • [44] Tran, L. T. (1990). Density estimation for time series by histograms. J. Statist. Plann. Inference 40 61-79.
  • [45] Yu, B. (1993). Density estimation in the L norm for dependent data with applications to the Gibbs sampler. Ann. Statist. 21 711-735.