The Annals of Mathematical Statistics

On Transient Markov Chains with Application to the Uniqueness Problem for Markov Processes

Leo Breiman

Full-text: Open access

Abstract

We focus our attention herein on a Markov chain $x_0, x_1, \cdots$ with a countable number of states indexed by a subset I of the integers and with stationary transition probabilities $p_{ij}$, and explore the sets of states defined by: A transient set of states $C$ is said to be denumerably atomic if $P(x_n \varepsilon C i.o.) > 0$ and if for every infinite set $A \subset C$ we have $x_n \varepsilon C i.o.$ implies $x_n \varepsilon A i.o.$ with probability one (a.s.). Following Blackwell's basic paper [1] which introduced the systematic use of martingales into the study of Markov chains, we use the semi-martingale convergence theorem [2] to characterize denumerably atomic sets in terms of the bounded solutions of the inequality $$\phi(i) \leqq \sum_{j \varepsilon I} p_{ij}\phi(j),\quad i \varepsilon I.$$ For chains whose state space contains a denumerably atomic set a convergence criterion for certain sums $\sum^\infty_{n = 0}f(x_n)$ is then developed. The application of this criterion to a restricted class of continuous parameter Markov processes gives simple necessary and sufficient conditions for the existence of a unique process satisfying given infinitesimal conditions. This last result illuminates the connection between the necessary and sufficient conditions given by Feller [3] for uniqueness and the simpler conditions for birth and death processes given recently by Dobrusin [4], more recently by Karlin and McGregor [5], and by Reuter and Lederman [6] (see also [7]).

Article information

Source
Ann. Math. Statist., Volume 28, Number 2 (1957), 499-503.

Dates
First available in Project Euclid: 27 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aoms/1177706979

Digital Object Identifier
doi:10.1214/aoms/1177706979

Mathematical Reviews number (MathSciNet)
MR88101

Zentralblatt MATH identifier
0078.31703

JSTOR
links.jstor.org

Citation

Breiman, Leo. On Transient Markov Chains with Application to the Uniqueness Problem for Markov Processes. Ann. Math. Statist. 28 (1957), no. 2, 499--503. doi:10.1214/aoms/1177706979. https://projecteuclid.org/euclid.aoms/1177706979


Export citation