The Annals of Probability

On density estimation from ergodic processes

Terrence M. Adams and Andrew B. Nobel

Full-text: Open access


We consider the problem of $L_p$-consistent density estimation from the initial segments of strongly dependent processes. It is shown that no procedure can consistently estimate the one-dimensional marginal density of every stationary ergodic process for which such a density exists. A similar result is established for the problem of estimating the support of the marginal distribution of an ergodic process.

Article information

Ann. Probab., Volume 26, Number 2 (1998), 794-804.

First available in Project Euclid: 31 May 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation 60G17: Sample path properties 60G10: Stationary processes

Density estimation ergodic processes cutting and stacking counter-example


Adams, Terrence M.; Nobel, Andrew B. On density estimation from ergodic processes. Ann. Probab. 26 (1998), no. 2, 794--804. doi:10.1214/aop/1022855650.

Export citation


  • Ahmad, I. A. (1979). Strong consistency of density estimation by orthogonal series methods for dependent variables with applications. Ann. Inst. Statist. Math. 31 279-288.
  • Barron, A. R. (1985). Logically smooth density estimation. Technical Report 56, Dept. Statistics, Stanford Univ.
  • Castellana, J. V. and Leadbetter, M. R. (1986). On smoothed probability density estimation for stationary processes. Stochastic Process Appl. 21 179-193.
  • Devroye, L. and Gy ¨orfi, L. (1985). Nonparametric Density Estimation: The L1-View. Wiley, New York.
  • Devroye, L. and Gy ¨orfi, L. (1990). No empirical probability measure can converge in the total variation sense for all distributions. Ann. Statist. 18 1496-1499.
  • Friedman, N. A. (1970). Introduction to Ergodic Theory. Van Nostrand Reinhold, New York.
  • Gy ¨orfi, L. (1981). Strongly consistent density estimate from ergodic sample. J. Multivariate Anal. 11 81-84.
  • Gy ¨orfi, L. and Lugosi, G. (1992). Kernel density estimation from ergodic sample is not universally consistent. Comput. Statist. Data Anal. 14 437-442.
  • Gy ¨orfi, L. and Masry, E. (1990). The L1 and L2 strong consistency of recursive kernel density estimation from dependent samples. IEEE Trans. Inform. Theory 36 531-539.
  • Gy ¨orfi, L., H¨ardle, W., Sarda, P. and Vieu, P. (1989). Nonparametric Curve Estimation from Time Series. Springer, Berlin.
  • Hall, P. and Hart, J. D. (1990). Convergence rates in density estimation for data from infiniteorder moving average processes. Probab. Theory Related Fields 87 253-274.
  • Nobel, A. B., Morvai, G. and Kulkarni, S. (1998). Density estimation from an individual numerical sequence. IEEE Trans. Inform. Theory 44 537-541.
  • Ornstein, D. and Weiss, B. (1990). How sampling reveals a process. Ann. Probab. 18 905-930.
  • Rosenblatt, M. (1970). Density estimates and Markov sequences. In Nonparametric Techniques in Statistical Inference (M. Puri, ed.). Cambridge Univ. Press.
  • Roussas, G. (1967). Nonparametric estimation in Markov processes. Ann. Inst. Statist. Math. 21 73-87.
  • Shields, P. C. (1991). Cutting and stacking: a method for constructing stationary processes. IEEE Trans. Inform. Theory 37 1605-1617.
  • Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman and Hall, London.
  • von Neumann, J. (1932). Zur Operatorenmethode in der klassichen Mechanik. Ann. Math. 33 587-642.
  • Yakowitz, S. (1989). Nonparametric density and regression estimation for Markov sequences without mixing assumptions. J. Multivariate Anal. 30 124-136.
  • Yakowitz, S. and Heyde, C (1997). Long-range dependency effects with implications for forecasting and queueing inference. Unpublished manuscript.