Asymptotic behaviour of stationary distributions for countable Markov chains, with some applications

Abstract

Let $\{Z_n,n\ge 0\}$ be an aperiodic irreducible recurrent (not necessarily positive recurrent) Markov chain taking values on a countable unbounded subset $S$ of $R^d$, $\pi (\cdot )$ its invariant measure and $f$ is a non-negative function defined on $S$. We first find sufficient conditions under which ${\int }_S^{}f\left(z\right)\pi (dz)=\mathrm{\infty }$ (the corresponding result for the finiteness of ${\int }_S^{}f\left(z\right)\pi (dz)$ was obtained by Tweedie). Then we obtain lower and upper bounds for the values of the invariant measure $\pi$ on the subsets $B$ of $S$, that is, $\pi \left(B\right)$. These bounds are expressed in terms of first passage probabilities and the first exit time from $B$. We also show how to estimate the latter quantities using sub- or supermartingale techniques. The results are finally illustrated for driftless reflected random walks in ${\mathbb{Z}}_{+}^2$ and for Markov chains on non-negative reals with asymptotically small drift of Lamperti type. In both cases we obtain very precise information on the asymptotic behaviour of their stationary measures.