The Annals of Statistics

On bandwidth choice for density estimation with dependent data

Peter Hall, Soumendra Nath Lahiri, and Young K. Truong

Full-text: Open access


We address the empirical bandwidth choice problem in cases where the range of dependence may be virtually arbitrarily long. Assuming that the observed data derive from an unknown function of a Gaussian process, it is argued that, unlike more traditional contexts of statistical inference, in density estimation there is no clear role for the classical distinction between short- and long-range dependence. Indeed, the "boundaries" that separate different modes of behaviour for optimal bandwidths and mean squared errors are determined more by kernel order than by traditional notions of strength of dependence, for example, by whether or not the sum of the covariances converges. We provide surprising evidence that, even for some strongly dependent data sequences, the asymptotically optimal bandwidth for independent data is a good choice. A plug-in empirical bandwidth selector based on this observation is suggested. We determine the properties of this choice for a wide range of different strengths of dependence. Properties of cross-validation are also addressed.

Article information

Ann. Statist., Volume 23, Number 6 (1995), 2241-2263.

First available in Project Euclid: 15 October 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation
Secondary: 62M10: Time series, auto-correlation, regression, etc. [See also 91B84]

Bandwidth choice cross-validation density estimation Gaussian process integrated squared error kernel methods long-range dependence mean integrated squared error plug-in rule short-range dependence window width


Hall, Peter; Lahiri, Soumendra Nath; Truong, Young K. On bandwidth choice for density estimation with dependent data. Ann. Statist. 23 (1995), no. 6, 2241--2263. doi:10.1214/aos/1034713655.

Export citation


  • BERAN, J. 1992. Statistical methods for data with long-range dependence with discussion. Statist. Sci 7 404 427. Z.
  • BICKEL, P. and RITOV, Y. 1988. Estimating integrated squared density derivatives: sharp best order of convergence estimates. Sankhy a Ser. A 50 381 393. Z.
  • BOWMAN, A. 1984. An alternative method of cross-validation for the smoothing of density estimates. Biometrika 71 353 360. Z.
  • CASTELLANA, J. V. 1989. Integrated consistency of smoothed probability density estimators for stationary sequences. Stochastic Process. Appl. 33 335 346. Z.
  • CASTELLANA, J. V. and LEADBETTER, M. R. 1986. On smoothed probability density estimation for stationary processes. Stochastic Process. Appl. 21 179 193. Z.
  • GRANGER, C. W. J. and JOy EUX, R. 1980. An introduction to long-memory time series models and fractional differencing. J. Time Ser. Anal. 1 15 29. Z.
  • Gy ORFI, L. and MASRY, E. 1990. The L and L strong consistency of recursive kernel density ¨ 1 2 estimation from dependent samples. IEEE Trans. Inform. Theory 36 531 539. Z.
  • HALL, P. 1983. Large sample optimality of least squares cross-validation in density estimation. Ann. Statist. 11 1156 1174. Z.
  • HALL, P. and HART, J. D. 1990. Convergence rates for density estimation for data from infinite order moving average sequences. Probab. Theory Related Fields 87 253 274. Z.
  • HALL, P., SHEATHER, S. J., JONES, M. C. and MARRON, J. S. 1991. On optimal data-based bandwidth selection in kernel density estimation. Biometrika 78 263 269. Z.
  • HART, J. D. 1984. Efficiency of a kernel density estimator under an autoregressive dependence model. J. Amer. Statist. Assoc. 79 110 117. Z.
  • HART, J. D. 1987. Kernel smoothing when the observations are correlated. Technical Report 35, Dept. Statistics, Texas A & M Univ. Z.
  • HART, J. D. 1991. Kernel regression estimation with time series errors. J. Roy. Statist. Soc. Ser. B 53 173 187. Z.
  • HART, J. D. and VIEU, P. 1990. Data-driven bandwidth choice for density estimation based on dependent data. Ann. Statist. 18 873 890. Z.
  • HASLETT, J. and RAFTERY, A. E. 1989. Space-time modelling with long-memory dependence: Z. assessing Ireland's wind power resource with discussions. J. Roy. Statist. Soc. Ser. C 38 1 50.Z.
  • HOSKING, J. R. M. 1981. Fractional differencing. Biometrika 68 165 176. Z.
  • HURST, H. E. 1951. Long-term storage capacity of reservoirs. Transactions of the American Society of Civil Engineers 116 770 779. Z.
  • JONES, M. C. 1991. The role of ISE and MISE in density estimation. Statist. Probab. Lett. 12 51 56. Z.
  • MAMMEN, E. 1990. A short note on optimal bandwidth selection for kernel estimators. Statist. Probab. Lett. 9 23 25. Z.
  • MELOCHE, J. 1990. Asy mptotic behaviour of the mean integrated squared error or kernel density estimators for dependent observations. Canad. J. Statist. 18 205 211.
  • NGUy EN, H. T. 1979. Density estimation in a continuous-time stationary Markov process. Ann. Statist. 7 341 348. Z.
  • PARK, B. U. and MARRON, J. S. 1990. Comparison of data-driven bandwidth selectors. J. Amer. Statist. Assoc. 85 66 72. Z.
  • ROSENBLATT, M. 1970. Density estimates and Markov sequences. In Nonparametric Techniques Z. in Statistical Inference M. L. Puri, ed. 199 210. Cambridge Univ. Press. Z.
  • ROUSSAS, G. G. 1969. Nonparametric estimation in Markov processes. Ann. Inst. Statist. Math. 21 73 87. Z.
  • ROUSSAS, G. G. 1988. Nonparametric estimation in mixing sequences of random variables. J. Statist. Plann. Inference 18 135 139. Z.
  • ROUSSAS, G. G. 1990a. Asy mptotic normality of the kernel estimate under dependence conditions: application to hazard rate. J. Statist. Plann. Inference 25 81 104. Z.
  • ROUSSAS, G. G. 1990b. Exact rates of almost sure convergence of a recursive estimate of a probability density function: application to regression and hazard rate estimation. Journal of Nonparametric Statistics 1 171 195. Z.
  • ROUSSAS, G. G. 1991. Kernel estimates under association: strong uniform consistency. Statist. Probab. Lett. 12 393 403. Z.
  • ROUSSAS, G. G. and IOANNIDES, D. 1987. Note on the uniform convergence of density estimates for mixing random variables. Statist. Probab. Lett. 5 179 285. Z.
  • RUDEMO, M. 1982. Empirical choice of histograms and kernel density estimators. Scand. J. Statist. 9 65 78. Z.
  • SCOTT, D. W. 1988. Discussion of ``How far are automatically chosen regression smoothing parameters from their optimum?'' by W. Hardle, P. Hall and J. S. Marron. J. Amer. ¨ Statist. Assoc. 83 96 98. Z.
  • SILVERMAN, B. W. 1986. Density Estimation for Statistics and Data Analy sis. Chapman and Hall, London. Z.
  • STONE, C. J. 1984. An asy mptotically optimal window selection rule for kernel density estimates. Ann. Statist. 12 1285 1297. Z.
  • TRAN, L. T. 1989. The L performance of kernel density estimates under dependence. Canad. J. 1 Statist. 17 197 208. Z.
  • TRAN, L. T. 1990a. Kernel density estimation under dependence. Statist. Probab. Lett. 10 193 201. Z.
  • TRAN, L. T. 1990b. Kernel density estimation on random fields. J. Multivariate Anal. 34 37 53. Z.
  • YAKOWITZ, S. 1985. Nonparametric density estimation, prediction and regression for Markov sequences. J. Amer. Statist. Assoc. 80 215 221. Z.
  • YAKOWITZ, S. 1989. Nonparametric density and regression estimation for Markov sequences without mixing assumptions. J. Multivariate Anal. 30 124 136. Z.
  • YU, B. 1993. Density estimation in the L norm for dependent data, with applications to the Gibbs sampler. Ann. Statist. 21 711 735.