Tree formulas, mean first passage times and Kemeny’s constant of a Markov chain

Abstract

This paper offers some probabilistic and combinatorial insights into tree formulas for the Green function and hitting probabilities of Markov chains on a finite state space. These tree formulas are closely related to loop-erased random walks by Wilson’s algorithm for random spanning trees, and to mixing times by the Markov chain tree theorem. Let $m_{ij}$ be the mean first passage time from $i$ to $j$ for an irreducible chain with finite state space $S$ and transition matrix $(p_{ij};i,j\in S)$. It is well known that $m_{jj}=1/\pi_{j}=\Sigma^{(1)}/\Sigma_{j}$, where $\pi$ is the stationary distribution for the chain, $\Sigma_{j}$ is the tree sum, over $n^{n-2}$ trees $\mathbf{t}$ spanning $S$ with root $j$ and edges $i\rightarrow k$ directed towards $j$, of the tree product $\prod_{i\rightarrow k\in\mathbf{t}}p_{ik}$, and $\Sigma^{(1)}:=\sum_{j\in S}\Sigma_{j}$. Chebotarev and Agaev (Linear Algebra Appl. 356 (2002) 253–274) derived further results from Kirchhoff’s matrix tree theorem. We deduce that for $i\ne j$, $m_{ij}=\Sigma_{ij}/\Sigma_{j}$, where $\Sigma_{ij}$ is the sum over the same set of $n^{n-2}$ spanning trees of the same tree product as for $\Sigma_{j}$, except that in each product the factor $p_{kj}$ is omitted where $k=k(i,j,\mathbf{t})$ is the last state before $j$ in the path from $i$ to $j$ in $\mathbf{t}$. It follows that Kemeny’s constant $\sum_{j\in S}m_{ij}/m_{jj}$ equals $\Sigma^{(2)}/\Sigma^{(1)}$, where $\Sigma^{(r)}$ is the sum, over all forests $\mathbf{f}$ labeled by $S$ with $r$ directed trees, of the product of $p_{ij}$ over edges $i\rightarrow j$ of $\mathbf{f}$. We show that these results can be derived without appeal to the matrix tree theorem. A list of relevant literature is also reviewed.