The Annals of Applied Probability

State-Dependent Criteria for Convergence of Markov Chains

Sean P. Meyn and R. L. Tweedie

Full-text: Open access

Abstract

The standard Foster-Lyapunov approach to establishing recurrence and ergodicity of Markov chains requires that the one-step mean drift of the chain be negative outside some appropriately finite set. Malyshev and Men'sikov developed a refinement of this approach for countable state space chains, allowing the drift to be negative after a number of steps depending on the starting state. We show that these countable space results are special cases of those in the wider context of $\varphi$-irreducible chains, and we give sample-path proofs natural for such chains which are rather more transparent than the original proofs of Malyshev and Men'sikov. We also develop an associated random-step approach giving similar conclusions. We further find state-dependent drift conditions sufficient to show that the chain is actually geometrically ergodic; that is, it has $n$-step transition probabilities which converge to their limits geometrically quickly. We apply these methods to a model of antibody activity and to a nonlinear threshold autoregressive model; they are also applicable to the analysis of complex queueing models.

Article information

Source
Ann. Appl. Probab., Volume 4, Number 1 (1994), 149-168.

Dates
First available in Project Euclid: 19 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1177005204

Digital Object Identifier
doi:10.1214/aoap/1177005204

Mathematical Reviews number (MathSciNet)
MR1258177

Zentralblatt MATH identifier
0803.60060

JSTOR
links.jstor.org

Subjects
Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)

Keywords
Foster's criterion irreducible Markov processes Lyapunov functions ergodicity geometric ergodicity recurrence Harris recurrence invasion models autoregressions networks of queues

Citation

Meyn, Sean P.; Tweedie, R. L. State-Dependent Criteria for Convergence of Markov Chains. Ann. Appl. Probab. 4 (1994), no. 1, 149--168. doi:10.1214/aoap/1177005204. https://projecteuclid.org/euclid.aoap/1177005204


Export citation