The Annals of Applied Probability

Stochastic approximation of quasi-stationary distributions on compact spaces and applications

Michel Benaim, Bertrand Cloez, and Fabien Panloup

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


As a continuation of a recent paper, dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact metric space killed in finite time. The idea is to run the process until extinction and then to bring it back to life at a position randomly chosen according to the (possibly weighted) empirical occupation measure of its past positions. General conditions are given ensuring the convergence of this measure to the quasi-stationary distribution of the chain. We then apply this method to the numerical approximation of the quasi-stationary distribution of a diffusion process killed on the boundary of a compact set. Finally, the sharpness of the assumptions is illustrated through the study of the algorithm in a nonirreducible setting.

Article information

Ann. Appl. Probab., Volume 28, Number 4 (2018), 2370-2416.

Received: December 2016
Revised: September 2017
First available in Project Euclid: 9 August 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 65C20: Models, numerical methods [See also 68U20] 60B12: Limit theorems for vector-valued random variables (infinite- dimensional case) 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 34F05: Equations and systems with randomness [See also 34K50, 60H10, 93E03] 60J20: Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) [See also 90B30, 91D10, 91D35, 91E40] 60J60: Diffusion processes [See also 58J65]

Quasi-stationary distributions stochastic approximation reinforced random walks random perturbations of dynamical systems extinction rate Euler scheme


Benaim, Michel; Cloez, Bertrand; Panloup, Fabien. Stochastic approximation of quasi-stationary distributions on compact spaces and applications. Ann. Appl. Probab. 28 (2018), no. 4, 2370--2416. doi:10.1214/17-AAP1360.

Export citation


  • [1] Aldous, D., Flannery, B. and Palacios, J. (1988). Two applications of urn processes: The fringe analysis of search trees and the simulation of quasi-stationary distributions of Markov chains. Probab. Engrg. Inform. Sci. 2 293–307.
  • [2] Bansaye, V., Méléard, S. and Richard, M. (2016). Speed of coming down from infinity for birth-and-death processes. Adv. in Appl. Probab. 48 1183–1210.
  • [3] Bass, R. F. (1998). Diffusions and Elliptic Operators. Springer, New York.
  • [4] Ben-Ari, I. and Pinsky, R. G. (2007). Spectral analysis of a family of second-order elliptic operators with nonlocal boundary condition indexed by a probability measure. J. Funct. Anal. 251 122–140.
  • [5] Benaïm, M. (1999). Dynamics of stochastic approximation algorithms. In Séminaire de Probabilités, XXXIII. Lecture Notes in Math. 1709 1–68. Springer, Berlin.
  • [6] Benaïm, M. and Cloez, B. (2015). A stochastic approximation approach to quasi-stationary distributions on finite spaces. Electron. Commun. Probab. 20 no. 37.
  • [7] Benaïm, M. and Hirsch, M. W. (1996). Asymptotic pseudotrajectories and chain recurrent flows, with applications. J. Dynam. Differential Equations 8 141–176.
  • [8] Benaïm, M., Ledoux, M. and Raimond, O. (2002). Self-interacting diffusions. Probab. Theory Related Fields 122 1–41.
  • [9] Berglund, N. and Landon, D. (2012). Mixed-mode oscillations and interspike interval statistics in the stochastic FitzHugh–Nagumo model. Nonlinearity 25 2303–2335.
  • [10] Bieniek, M., Burdzy, K. and Finch, S. (2012). Non-extinction of a Fleming–Viot particle model. Probab. Theory Related Fields 153 293–332.
  • [11] Bieniek, M., Burdzy, K. and Pal, S. (2012). Extinction of Fleming–Viot-type particle systems with strong drift. Electron. J. Probab. 17 no. 11.
  • [12] Blanchet, J., Glynn, P. and Zheng, S. (2016). Analysis of a stochastic approximation algorithm for computing quasi-stationary distributions. Adv. in Appl. Probab. 48 792–811.
  • [13] Bouleau, N. and Lépingle, D. (1994). Numerical Methods for Stochastic Processes. Wiley, New York.
  • [14] Burdzy, K., Hołyst, R. and March, P. (2000). A Fleming–Viot particle representation of the Dirichlet Laplacian. Comm. Math. Phys. 214 679–703.
  • [15] Cattiaux, P., Collet, P., Lambert, A., Martínez, S., Méléard, S. and San Martín, J. (2009). Quasi-stationary distributions and diffusion models in population dynamics. Ann. Probab. 37 1926–1969.
  • [16] Champagnat, N. and Villemonais, D. (2016). Exponential convergence to quasi-stationary distribution and $Q$-process. Probab. Theory Related Fields 164 243–283.
  • [17] Cloez, B. and Thai, M.-N. (2016). Fleming–Viot processes: Two explicit examples. ALEA Lat. Am. J. Probab. Math. Stat. 13 337–356.
  • [18] Cloez, B. and Thai, M.-N. (2016). Quantitative results for the Fleming–Viot particle system and quasi-stationary distributions in discrete space. Stochastic Process. Appl. 126 680–702.
  • [19] Collet, P., Martínez, S., Méléard, S. and San Martín, J. (2011). Quasi-stationary distributions for structured birth and death processes with mutations. Probab. Theory Related Fields 151 191–231.
  • [20] Del Moral, P. and Guionnet, A. (1999). On the stability of measure valued processes with applications to filtering. C. R. Acad. Sci. Paris, Sér. I Math. 329 429–434.
  • [21] Del Moral, P. and Miclo, L. (2000). A Moran particle system approximation of Feynman–Kac formulae. Stochastic Process. Appl. 86 193–216.
  • [22] Dudley, R. M. (2002). Real Analysis and Probability. Cambridge Studies in Advanced Mathematics 74. Cambridge Univ. Press, Cambridge. Revised reprint of the 1989 original.
  • [23] Duflo, M. (2000). Random Iterative Models. Springer, Berlin.
  • [24] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence. Wiley, New York.
  • [25] Ferrari, P. A., Kesten, H., Martinez, S. and Picco, P. (1995). Existence of quasi-stationary distributions. A renewal dynamical approach. Ann. Probab. 23 501–521.
  • [26] Ferrari, P. A. and Marić, N. (2007). Quasi stationary distributions and Fleming–Viot processes in countable spaces. Electron. J. Probab. 12 684–702.
  • [27] Gobet, E. (2000). Weak approximation of killed diffusion using Euler schemes. Stochastic Process. Appl. 87 167–197.
  • [28] Gong, G. L., Qian, M. P. and Zhao, Z. X. (1988). Killed diffusions and their conditioning. Probab. Theory Related Fields 80 151–167.
  • [29] Grigorescu, I. and Kang, M. (2012). Immortal particle for a catalytic branching process. Probab. Theory Related Fields 153 333–361.
  • [30] Lamberton, D. and Pagès, G. (2008). A penalized bandit algorithm. Electron. J. Probab. 13 341–373.
  • [31] Lamberton, D., Pagès, G. and Tarrès, P. (2004). When can the two-armed bandit algorithm be trusted? Ann. Appl. Probab. 14 1424–1454.
  • [32] Lemaire, V. (2007). An adaptive scheme for the approximation of dissipative systems. Stochastic Process. Appl. 117 1491–1518.
  • [33] Lemaire, V. and Menozzi, S. (2010). On some non asymptotic bounds for the Euler scheme. Electron. J. Probab. 15 1645–1681.
  • [34] Méléard, S. and Villemonais, D. (2012). Quasi-stationary distributions and population processes. Probab. Surv. 9 340–410.
  • [35] Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markovian processes. III. Foster–Lyapunov criteria for continuous-time processes. Adv. in Appl. Probab. 25 518–548.
  • [36] Oçafrain, W. and Villemonais, D. (2016). Non-failable approximation method for conditioned distributions. Preprint. Available at arXiv:1606.08978.
  • [37] Panloup, F. (2008). Recursive computation of the invariant measure of a stochastic differential equation driven by a Lévy process. Ann. Appl. Probab. 18 379–426.
  • [38] Pemantle, R. (2007). A survey of random processes with reinforcement. Probab. Surv. 4 1–79.
  • [39] Pinsky, R. G. (1995). Positive Harmonic Functions and Diffusion. Cambridge Studies in Advanced Mathematics 45. Cambridge Univ. Press, Cambridge.
  • [40] Pollock, M., Fearnhead, P., Johansen, A. M. and Roberts, G. O. (2016). The scalable Langevin exact algorithm: Bayesian inference for big data. Preprint. Available at arXiv:1609.03436.
  • [41] Rachev, S. T., Klebanov, L. B., Stoyanov, S. V. and Fabozzi, F. J. (2013). The Methods of Distances in the Theory of Probability and Statistics. Springer, New York.
  • [42] Villemonais, D. (2011). Interacting particle systems and Yaglom limit approximation of diffusions with unbounded drift. Electron. J. Probab. 16 1663–1692.
  • [43] Villemonais, D. (2014). General approximation method for the distribution of Markov processes conditioned not to be killed. ESAIM Probab. Stat. 18 441–467.