Electronic Journal of Probability

Fluctuations of the empirical measure of freezing Markov chains

Florian Bouguet and Bertrand Cloez

Full-text: Open access


In this work, we consider a finite-state inhomogeneous-time Markov chain whose probabilities of transition from one state to another tend to decrease over time. This can be seen as a cooling of the dynamics of an underlying Markov chain. We are interested in the long time behavior of the empirical measure of this freezing Markov chain. Some recent papers provide almost sure convergence and convergence in distribution in the case of the freezing speed $n^{-\theta }$, with different limits depending on $\theta <1,\theta =1$ or $\theta >1$. Using stochastic approximation techniques, we generalize these results for any freezing speed, and we obtain a better characterization of the limit distribution as well as rates of convergence and functional convergence.

Article information

Electron. J. Probab., Volume 23 (2018), paper no. 2, 31 pp.

Received: 11 May 2017
Accepted: 15 December 2017
First available in Project Euclid: 16 January 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces) 60J25: Continuous-time Markov processes on general state spaces 60F05: Central limit and other weak theorems

Markov chain long-time behavior piecewise-deterministic Markov process Ornstein-Uhlenbeck process asymptotic pseudotrajectory

Creative Commons Attribution 4.0 International License.


Bouguet, Florian; Cloez, Bertrand. Fluctuations of the empirical measure of freezing Markov chains. Electron. J. Probab. 23 (2018), paper no. 2, 31 pp. doi:10.1214/17-EJP130. https://projecteuclid.org/euclid.ejp/1516093310

Export citation


  • [1] Y. Bakhtin and T. Hurth, Invariant densities for dynamical systems with random switching, Nonlinearity 25 (2012), no. 10, 2937–2952.
  • [2] M. Benaïm, Vertex-reinforced random walks and a conjecture of Pemantle, Ann. Probab. 25 (1997), no. 1, 361–392.
  • [3] M. Benaïm, Dynamics of stochastic approximation algorithms, Séminaire de Probabilités, XXXIII, Lecture Notes in Math., vol. 1709, Springer, Berlin, 1999, pp. 1–68.
  • [4] M. Benaïm, F. Bouguet, and B. Cloez, Ergodicity of inhomogeneous markov chains through asymptotic pseudotrajectories, Ann. Appl. Probab. 27 (2017), no. 5, 3004–3049.
  • [5] M. Benaïm and B. Cloez, A stochastic approximation approach to quasi-stationary distributions on finite spaces, Electron. Commun. Probab. 20 (2015), no. 37, 14.
  • [6] M. Benaïm and M. W. Hirsch, Asymptotic pseudotrajectories and chain recurrent flows, with applications, J. Dynam. Differential Equations 8 (1996), no. 1, 141–176.
  • [7] M. Benaïm, S. Le Borgne, F. Malrieu, and P.-A. Zitt, Quantitative ergodicity for some switched dynamical systems, Electron. Commun. Probab. 17 (2012), no. 56, 14.
  • [8] M. Benaïm, Qualitative properties of certain piecewise deterministic Markov processes, Ann. Inst. Henri Poincaré Probab. Stat. 51 (2015), no. 3, 1040–1075.
  • [9] Joris Bierkens and Andrew Duncan, Limit theorems for the zig-zag process, Advances in Applied Probability 49 (2017), no. 3, 791–825.
  • [10] F. Bouguet, F. Malrieu, F. Panloup, C. Poquet, and J. Reygner, Long time behavior of Markov processes and beyond, ESAIM Proc. Surv. 51 (2015), 193–211.
  • [11] B. Cloez and M. Hairer, Exponential ergodicity for Markov processes with random switching, Bernoulli 21 (2015), no. 1, 505–536.
  • [12] C. Costantini and T. G. Kurtz, Diffusion approximation for transport processes with general reflection boundary conditions, Math. Models Methods Appl. Sci. 16 (2006), no. 5, 717–762.
  • [13] M. H. A. Davis, Markov models & optimization, vol. 49, CRC Press, 1993.
  • [14] Z. Dietz and S. Sethuraman, Occupation laws for some time-nonhomogeneous Markov chains, Electron. J. Probab. 12 (2007), no. 23, 661–683.
  • [15] R. L. Dobrušin, Limit theorems for a Markov chain of two states, Izvestiya Akad. Nauk SSSR. Ser. Mat. 17 (1953), 291–330.
  • [16] M. Duflo, Algorithmes stochastiques, Mathématiques & Applications (Berlin) [Mathematics & Applications], vol. 23, Springer-Verlag, Berlin, 1996.
  • [17] János Engländer and Stanislav Volkov, Turning a coin over instead of tossing it, Journal of Theoretical Probability (2016).
  • [18] S. N. Ethier and T. G. Kurtz, Markov processes, Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics, John Wiley & Sons, Inc., New York, 1986, Characterization and convergence.
  • [19] J. Fontbona, H. Guérin, and F. Malrieu, Quantitative estimates for the long-time behavior of an ergodic variant of the telegraph process, Adv. in Appl. Probab. 44 (2012), no. 4, 977–994.
  • [20] G. Fort, Central limit theorems for stochastic approximation with controlled Markov chain dynamics, ESAIM Probab. Stat. 19 (2015), 60–80.
  • [21] F. R. Gantmacher, The Theory of Matrices, vol. 2, Chelsea, New York, 1959.
  • [22] R. Gouet, Strong convergence of proportions in a multicolor Pólya urn, J. Appl. Probab. 34 (1997), no. 2, 426–435.
  • [23] N. Ikeda and S. Watanabe, Stochastic differential equations and diffusion processes, second ed., North-Holland Mathematical Library, vol. 24, North-Holland Publishing Co., Amsterdam; Kodansha, Ltd., Tokyo, 1989.
  • [24] G. L. Jones, On the Markov chain central limit theorem, Probab. Surv. 1 (2004), 299–320.
  • [25] V. N. Kolokoltsov, Markov processes, semigroups and generators, De Gruyter Studies in Mathematics, vol. 38, Walter de Gruyter & Co., Berlin, 2011.
  • [26] H. Kunita, Stochastic differential equations and stochastic flows of diffeomorphisms, École d’été de probabilités de Saint-Flour, XII—1982, Lecture Notes in Math., vol. 1097, Springer, Berlin, 1984, pp. 143–303.
  • [27] H. Kushner and G. Yin, Stochastic approximation and recursive algorithms and applications, vol. 35, Springer, 2003.
  • [28] M. Métivier and P. Priouret, Théorèmes de convergence presque sure pour une classe d’algorithmes stochastiques à pas décroissant, Probab. Theory Related Fields 74 (1987), no. 3, 403–428.
  • [29] M. Peligrad, Central limit theorem for triangular arrays of non-homogeneous Markov chains, Probab. Theory Related Fields 154 (2012), no. 3-4, 409–428.
  • [30] L. Saloff-Coste, Lectures on finite Markov chains, Lectures on probability theory and statistics (Saint-Flour, 1996), Lecture Notes in Math., vol. 1665, Springer, Berlin, 1997, pp. 301–413.
  • [31] S. Sethuraman and S. R. S. Varadhan, A martingale proof of Dobrushin’s theorem for non-homogeneous Markov chains, Electron. J. Probab. 10 (2005), no. 36, 1221–1235.