Abstract
Let $\{X_1, X_2,\cdots\}$ be a stationary process with probability densities $f(X_1, X_2,\cdots, X_n)$ with respect to Lebesgue measure or with respect to a Markov measure with a stationary transition measure. It is shown that the sequence of relative entropy densities $(1/n)\log f(X_1, X_2,\cdots, X_n)$ converges almost surely. This long-conjectured result extends the $L^1$ convergence obtained by Moy, Perez, and Kieffer and generalizes the Shannon-McMillan-Breiman theorem to nondiscrete processes. The heart of the proof is a new martingale inequality which shows that logarithms of densities are $L^1$ dominated.
Citation
Andrew R. Barron. "The Strong Ergodic Theorem for Densities: Generalized Shannon-McMillan-Breiman Theorem." Ann. Probab. 13 (4) 1292 - 1303, November, 1985. https://doi.org/10.1214/aop/1176992813
Information