## Bernoulli

• Bernoulli
• Volume 24, Number 3 (2018), 1942-1972.

### Tree formulas, mean first passage times and Kemeny’s constant of a Markov chain

#### Abstract

This paper offers some probabilistic and combinatorial insights into tree formulas for the Green function and hitting probabilities of Markov chains on a finite state space. These tree formulas are closely related to loop-erased random walks by Wilson’s algorithm for random spanning trees, and to mixing times by the Markov chain tree theorem. Let $m_{ij}$ be the mean first passage time from $i$ to $j$ for an irreducible chain with finite state space $S$ and transition matrix $(p_{ij};i,j\in S)$. It is well known that $m_{jj}=1/\pi_{j}=\Sigma^{(1)}/\Sigma_{j}$, where $\pi$ is the stationary distribution for the chain, $\Sigma_{j}$ is the tree sum, over $n^{n-2}$ trees $\mathbf{t}$ spanning $S$ with root $j$ and edges $i\rightarrow k$ directed towards $j$, of the tree product $\prod_{i\rightarrow k\in\mathbf{t}}p_{ik}$, and $\Sigma^{(1)}:=\sum_{j\in S}\Sigma_{j}$. Chebotarev and Agaev (Linear Algebra Appl. 356 (2002) 253–274) derived further results from Kirchhoff’s matrix tree theorem. We deduce that for $i\ne j$, $m_{ij}=\Sigma_{ij}/\Sigma_{j}$, where $\Sigma_{ij}$ is the sum over the same set of $n^{n-2}$ spanning trees of the same tree product as for $\Sigma_{j}$, except that in each product the factor $p_{kj}$ is omitted where $k=k(i,j,\mathbf{t})$ is the last state before $j$ in the path from $i$ to $j$ in $\mathbf{t}$. It follows that Kemeny’s constant $\sum_{j\in S}m_{ij}/m_{jj}$ equals $\Sigma^{(2)}/\Sigma^{(1)}$, where $\Sigma^{(r)}$ is the sum, over all forests $\mathbf{f}$ labeled by $S$ with $r$ directed trees, of the product of $p_{ij}$ over edges $i\rightarrow j$ of $\mathbf{f}$. We show that these results can be derived without appeal to the matrix tree theorem. A list of relevant literature is also reviewed.

#### Article information

Source
Bernoulli, Volume 24, Number 3 (2018), 1942-1972.

Dates
Revised: September 2016
First available in Project Euclid: 2 February 2018

Permanent link to this document
https://projecteuclid.org/euclid.bj/1517540464

Digital Object Identifier
doi:10.3150/16-BEJ916

Mathematical Reviews number (MathSciNet)
MR3757519

Zentralblatt MATH identifier
06839256

#### Citation

Pitman, Jim; Tang, Wenpin. Tree formulas, mean first passage times and Kemeny’s constant of a Markov chain. Bernoulli 24 (2018), no. 3, 1942--1972. doi:10.3150/16-BEJ916. https://projecteuclid.org/euclid.bj/1517540464

#### References

• [1] Abdesselam, A. (2004). The Grassmann–Berezin calculus and theorems of the matrix-tree type. Adv. in Appl. Math. 33 51–70.
• [2] Aldous, D. and Fill, J.A. (2002). Markov Chains and random walks on graphs. Available at http://www.stat.berkeley.edu/aldous/RWG/book.html.
• [3] Aldous, D.J. (1990). The random walk construction of uniform spanning trees and uniform labelled trees. SIAM J. Discrete Math. 3 450–465.
• [4] Anantharam, V. and Tsoucas, P. (1989). A proof of the Markov chain tree theorem. Statist. Probab. Lett. 8 189–192.
• [5] Avena, L. and Gaudilliere, A. (2013). On some random forests with determinantal roots. Preprint. Available at arXiv:1310.1723.
• [6] Bellman, R. (1997). Introduction to Matrix Analysis. Classics in Applied Mathematics 19. Philadelphia, PA: SIAM.
• [7] Benjamin, A.T. and Yerger, C.R. (2006). Combinatorial interpretations of spanning tree identities. Bull. Inst. Combin. Appl. 47 37–42.
• [8] Benjamini, I., Lyons, R., Peres, Y. and Schramm, O. (2001). Uniform spanning forests. Ann. Probab. 29 1–65.
• [9] Benjamini, I. and Schramm, O. (2001). Recurrence of distributional limits of finite planar graphs. Electron. J. Probab. 6 no. 23, 13 pp. (electronic).
• [10] Biane, P. (2015). Polynomials associated with finite Markov chains. In In Memoriam Marc Yor – Séminaire de Probabilités XLVII. Lecture Notes in Math. 2137 249–262. Cham: Springer.
• [11] Biane, P. and Chapuy, G. (2015). Laplacian matrices and spanning trees of tree graphs. Preprint. Available at arXiv:1505.04806.
• [12] Biggs, N. (1993). Algebraic Graph Theory, 2nd ed. Cambridge Mathematical Library. Cambridge: Cambridge Univ. Press.
• [13] Bǫcher, M. (1964). Introduction to Higher Algebra. New York: Dover Publications.
• [14] Boesch, F.T. and Prodinger, H. (1986). Spanning tree formulas and Chebyshev polynomials. Graphs Combin. 2 191–200.
• [15] Bogner, C. and Weinzierl, S. (2010). Feynman graph polynomials. Internat. J. Modern Phys. A 25 2585–2618.
• [16] Bott, R. and Mayberry, J.P. (1954). Matrices and trees. In Economic Activity Analysis 391–400. New York: Wiley.
• [17] Broder, A. (1989). Generating random spanning trees. In Proceedings of the 30th Annual Symposium on Foundations of Computer Science (SFCS’89) 442–447. Washington, DC: IEEE Computer Society.
• [18] Brooks, R.L., Smith, C.A.B., Stone, A.H. and Tutte, W.T. (1940). The dissection of rectangles into squares. Duke Math. J. 7 312–340.
• [19] Burton, R. and Pemantle, R. (1993). Local characteristics, entropy and limit theorems for spanning trees and domino tilings via transfer-impedances. Ann. Probab. 21 1329–1371.
• [20] Catral, M., Kirkland, S.J., Neumann, M. and Sze, N.-S. (2010). The Kemeny constant for finite homogeneous ergodic Markov chains. J. Sci. Comput. 45 151–166.
• [21] Cayley, A. (1889). A theorem on trees. Pure Appl. Math. Q. 23 376–378.
• [22] Chaiken, S. (1982). A combinatorial proof of the all minors matrix tree theorem. SIAM J. Algebr. Discrete Methods 3 319–329.
• [23] Chaiken, S. and Kleitman, D.J. (1978). Matrix tree theorems. J. Combin. Theory Ser. A 24 377–381.
• [24] Chang, Y. and Jan, Y.L. (2014). Markov loops in discrete spaces. Preprint. Available at arXiv:1402.1064.
• [25] Chebotarev, P. (2007). A graph theoretic interpretation of the mean first passage times. Preprint. Available at arXiv:Math/0701359.
• [26] Chebotarev, P. and Agaev, R. (2002). Forest matrices around the Laplacian matrix. Linear Algebra Appl. 356 253–274.
• [27] Chen, W.K. (1976). Applied Graph Theory, revised ed. Amsterdam: North-Holland.
• [28] Chung, K.L. (1960). Markov Chains with Stationary Transition Probabilities. Die Grundlehren der Mathematischen Wissenschaften 104. Berlin: Springer.
• [29] Cori, R. and Le Borgne, Y. (2003). The sand-pile model and Tutte polynomials. Adv. in Appl. Math. 30 44–52.
• [30] de la Harpe, P. (2012). The Fuglede–Kadison determinant, theme and variations. Preprint. Available at arXiv:1107.1059.
• [31] Dhar, D. (1990). Self-organized critical state of sandpile automaton models. Phys. Rev. Lett. 64 1613–1616.
• [32] de Tilière, B. (2014). Principal minors Pfaffian half-tree theorem. J. Combin. Theory Ser. A 124 1–40.
• [33] Doyle, P. (2009). The Kemeny constant of a Markov chain. Available at https://math.dartmouth.edu/~doyle/docs/kc/kc.pdf.
• [34] Durrett, R. (2010). Probability: Theory and Examples, 4th ed. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge: Cambridge Univ. Press.
• [35] Duval, A.M., Klivans, C.J. and Martin, J.L. (2009). Simplicial matrix-tree theorems. Trans. Amer. Math. Soc. 361 6073–6114.
• [36] Feller, W. (1968). An Introduction to Probability Theory and Its Applications. Vol. I, 3rd ed. New York: Wiley.
• [37] Forman, R. (1993). Determinants of Laplacians on graphs. Topology 32 35–46.
• [38] Gorodezky, I. and Pak, I. (2014). Generalized loop-erased random walks and approximate reachability. Random Structures Algorithms 44 201–223.
• [39] Grimmett, G. (2010). Probability on Graphs: Random Processes on Graphs and Lattices. Institute of Mathematical Statistics Textbooks 1. Cambridge: Cambridge Univ. Press.
• [40] Gursoy, B.B., Kirkland, S., Mason, O. and Sergeev, S. (2013). On the Markov chain tree theorem in the max algebra. Electron. J. Linear Algebra 26 15–27.
• [41] Gustafson, K. and Hunter, J.J. (2015). Why the Kemeny Time is a constant. Preprint. Available at arXiv:1510.00456.
• [42] Hunter, J.J. (2014). The role of Kemeny’s constant in properties of Markov chains. Comm. Statist. Theory Methods 43 1309–1321.
• [43] Hunter, J.J. (2016). Accurate calculations of stationary distributions and mean first passage times in Markov renewal processes and Markov chains. Spec. Matrices 4 151–175.
• [44] Járai, A.A. (2014). Sandpile models. Preprint. Available at arXiv:1401.0354.
• [45] Kalai, G. (1983). Enumeration of ${\mathbf{Q}}$-acyclic simplicial complexes. Israel J. Math. 45 337–351.
• [46] Kassel, A. (2015). Learning about critical phenomena from scribbles and sandpiles. In Modélisation Aléatoire et Statistique – Journées MAS 2014. ESAIM Proc. Surveys 51 60–73. Les Ulis: EDP Sci.
• [47] Kassel, A. and Kenyon, R. (2012). Random curves on surfaces induced from the Laplacian determinant. Preprint. Available at arXiv:1211.6974.
• [48] Kassel, A., Kenyon, R. and Wu, W. (2015). Random two-component spanning forests. Ann. Inst. Henri Poincaré Probab. Stat. 51 1457–1464.
• [49] Kassel, A. and Wilson, D.B. (2016). The looping rate and sandpile density of planar graphs. Amer. Math. Monthly 123 19–39.
• [50] Kasteleyn, P.W. (1967). Graph theory and crystal physics. In Graph Theory and Theoretical Physics 43–110. London: Academic Press.
• [51] Kelmans, A.K. and Chelnokov, V.M. (1974). A certain polynomial of a graph and graphs with an extremal number of trees. J. Combin. Theory Ser. B 16 197–214.
• [52] Kelner, J.A. and Ma̧dry, A. (2009). Faster generation of random spanning trees. In 2009 50th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2009) 13–21. Los Alamitos, CA: IEEE Computer Soc.
• [53] Kemeny, J.G. and Snell, J.L. (1960). Finite Markov Chains. The University Series in Undergraduate Mathematics. Princeton, NJ: D. Van Nostrand Co.
• [54] Kenyon, R. (2009). Lectures on dimers. In Statistical Mechanics. IAS/Park City Math. Ser. 16 191–230. Providence, RI: Amer. Math. Soc.
• [55] Kenyon, R. (2011). Spanning forests and the vector bundle Laplacian. Ann. Probab. 39 1983–2017.
• [56] Kenyon, R.W., Propp, J.G. and Wilson, D.B. (2000). Trees and matchings. Electron. J. Combin. 7 Research Paper 25, 34 pp. (electronic).
• [57] Kenyon, R.W. and Wilson, D.B. (2015). Spanning trees of graphs on surfaces and the intensity of loop-erased random walk on planar graphs. J. Amer. Math. Soc. 28 985–1030.
• [58] Kirchhoff, G. (1847). Über die Auflösung der Gleichungen, auf welche man bei der Untersuchung der linearen Verteilung galvanischer Ströme geführt wird. Ann. Phys. Chem. 72 497–508.
• [59] Kirchhoff, G. (1958). On the solution of the equations obtained from the investigation of the linear distribution of galvanic currents (translated by J.B. O’Toole). IRE Trans. Circuit Theory 5 4–7.
• [60] Kohler, H.-H. and Vollmerhaus, E. (1980). The frequency of cyclic processes in biological multistate systems. J. Math. Biol. 9 275–290.
• [61] Kozdron, M.J., Richards, L.M. and Stroock, D.W. (2013). Determinants, their applications to Markov processes, and a random walk proof of Kirchhoff’s matrix tree theorem. Preprint. Available at arXiv:1306.2059.
• [62] Lawler, G.F. (1999). Loop-erased random walk. In Perplexing Problems in Probability. Progress in Probability 44 197–217. Boston, MA: Birkhäuser.
• [63] Lawler, G.F. and Limic, V. (2010). Random Walk: A Modern Introduction. Cambridge Studies in Advanced Mathematics 123. Cambridge: Cambridge Univ. Press.
• [64] Leighton, F.T. and Rivest, R.L. (1983). Estimating a probability using finite memory. In Foundations of Computation Theory (Borgholm, 1983). Lecture Notes in Computer Science 158 255–269. Berlin: Springer.
• [65] Leighton, F.T. and Rivest, R.L. (1983). The Markov chain tree theorem. Technical Report, M.I.T Laboratory for Computer Science, MIT/LCS/TM-249.
• [66] Levene, M. and Loizou, G. (2002). Kemeny’s constant and the random surfer. Amer. Math. Monthly 109 741–745.
• [67] Levin, D.A., Peres, Y. and Wilmer, E.L. (2009). Markov Chains and Mixing Times. Providence, RI: Amer. Math. Soc.
• [68] Liu, C.J. and Chow, Y. (1981). Enumeration of forests in a graph. Proc. Amer. Math. Soc. 83 659–662.
• [69] Lovász, L. and Winkler, P. (1998). Mixing times. In Microsurveys in Discrete Probability (Princeton, NJ, 1997). DIMACS Ser. Discrete Math. Theoret. Comput. Sci. 41 85–133. Providence, RI: Amer. Math. Soc.
• [70] Lyons, R. (1998). A bird’s-eye view of uniform spanning trees and forests. In Microsurveys in Discrete Probability (Princeton, NJ, 1997). DIMACS Ser. Discrete Math. Theoret. Comput. Sci. 41 135–162. Providence, RI: Amer. Math. Soc.
• [71] Lyons, R. (2005). Asymptotic enumeration of spanning trees. Combin. Probab. Comput. 14 491–522.
• [72] Lyons, R. (2009). Random complexes and $l^{2}$-Betti numbers. J. Topol. Anal. 1 153–175.
• [73] Lyons, R. (2010). Identities and inequalities for tree entropy. Combin. Probab. Comput. 19 303–313.
• [74] Lyons, R. and Peres, Y. (2016). Probability on Trees and Networks. Cambridge: Cambridge Univ. Press. Available at http://pages.iu.edu/~rdlyons/.
• [75] Ma̧dry, A., Straszak, D. and Tarnawski, J. (2015). Fast generation of random spanning trees and the effective resistance metric. In Proceedings of the Twenty-Sixth Annual ACM-SIAM Symposium on Discrete Algorithms 2019–2036. Philadelphia, PA: SIAM.
• [76] Majumdar, S.N. and Dhar, D. (1992). Equivalence between the Abelian sandpile model and the $q\rightarrow0$ limit of the Potts model. Physica A: Statistical Mechanics and Its Applications 185 129–145.
• [77] Marchal, P. (1999). Cycles hamiltoniens aléatoires et mesures d’occupation invariantes par une action de groupe. C. R. Acad. Sci. Paris Sér. I Math. 329 883–886.
• [78] Marchal, P. (2000). Loop-erased random walks, spanning trees and Hamiltonian cycles. Electron. Commun. Probab. 5 39–50 (electronic).
• [79] Masbaum, G. and Vaintrob, A. (2002). A new matrix-tree theorem. Int. Math. Res. Not. 27 1397–1426.
• [80] Meyer, C. (2000). Matrix Analysis and Applied Linear Algebra. Philadelphia, PA: SIAM.
• [81] Minoux, M. (1997). Bideterminants, arborescences and extension of the matrix-tree theorem to semirings. Discrete Math. 171 191–200.
• [82] Moon, J.W. (1970). Counting Labelled Trees. From Lectures Delivered to the Twelfth Biennial Seminar of the Canadian Mathematical Congress (Vancouver) 1969. Montreal, Que.: Canadian Mathematical Congress.
• [83] Orlin, J.B. (1978). Line-digraphs, arborescences, and theorems of Tutte and Knuth. J. Combin. Theory Ser. B 25 187–198.
• [84] Pak, I. and Postnikov, A. (1994). Enumeration of spanning trees of graphs. Available at http://math.mit.edu/~apost/papers/tree.ps.
• [85] Pemantle, R. (1991). Choosing a spanning tree for the integer lattice uniformly. Ann. Probab. 19 1559–1574.
• [86] Pitman, J. Enumerations and expectations related to the matrix tree expansion of a determinant. In preparation.
• [87] Pitman, J. (2001/2002). Random mappings, forests, and subsets associated with Abel–Cayley–Hurwitz multinomial expansions. Sém. Lothar. Combin. 46 Art. B46h, 45 pp. (electronic).
• [88] Pitman, J. (2002). Forest volume decompositions and Abel–Cayley–Hurwitz multinomial expansions. J. Combin. Theory Ser. A 98 175–191.
• [89] Pitman, J.W. (1977). Occupation measures for Markov chains. Adv. in Appl. Probab. 9 69–86.
• [90] Pokarowski, P. (1999). Directed forests with application to algorithms related to Markov chains. Appl. Math. (Warsaw) 26 395–414.
• [91] Propp, J.G. and Wilson, D.B. (1998). How to get a perfectly random sample from a generic Markov chain and generate a random spanning tree of a directed graph. J. Algorithms 27 170–217.
• [92] Runge, F. and Sachs, H. (1974). Berechnung der Anzahl der Gerüste von Graphen und Hypergraphen mittels deren Spektren. Math. Balkanica 4 529–536.
• [93] Sahi, S. (2014). Harmonic vectors and matrix tree theorems. J. Comb. 5 195–202.
• [94] Seneta, E. (2006). Non-negative Matrices and Markov Chains. Springer Series in Statistics. New York: Springer.
• [95] Shubert, B.O. (1975). A flow-graph formula for the stationary distribution of a Markov chain. IEEE Trans. Syst. Man Cybern. Syst. SMC-5 565–566.
• [96] Sokal, A.D. (2005). The multivariate Tutte polynomial (alias Potts model) for graphs and matroids. In Surveys in Combinatorics 2005. London Mathematical Society Lecture Note Series 327 173–226. Cambridge: Cambridge Univ. Press.
• [97] Solberg, J.J. (1974/1975). A graph theoretic formula for the steady state distribution of finite Markov processes. Manage. Sci. 21 1040–1048.
• [98] Stanley, R.P. (1999). Enumerative Combinatorics. Vol. 2. Cambridge Studies in Advanced Mathematics 62. Cambridge: Cambridge Univ. Press.
• [99] Stroock, D.W. (2014). An Introduction to Markov Processes, 2nd ed. Graduate Texts in Mathematics 230. Heidelberg: Springer.
• [100] Temperley, H.N.V. (1964). On the mutual cancellation of cluster integrals in Mayer’s fugacity series. Proc. Phys. Soc. 83 3–16.
• [101] Temperley, H.N.V. (1972). The enumeration of graphs on large periodic lattices. In Combinatorics (Proc. Conf. Combinatorial Math., Math. Inst., Oxford, 1972) 285–294. Southend-on-Sea: Inst. Math. Appl.
• [102] Temperley, H.N.V. (1974). Enumeration of graphs on a large periodic lattice. In Combinatorics (Proc. British Combinatorial Conf., Univ. Coll. Wales, Aberystwyth, 1973). London Math. Soc. Lecture Note Ser. 13 155–159. London: Cambridge Univ. Press.
• [103] Temperley, H.N.V. and Fisher, M.E. (1961). Dimer problem in statistical mechanics – an exact result. Philos. Mag. (8) 6 1061–1063.
• [104] Tutte, W.T. (1948). The dissection of equilateral triangles into equilateral triangles. Math. Proc. Cambridge Philos. Soc. 44 463–482.
• [105] Tutte, W.T. (2001). Graph Theory. Encyclopedia of Mathematics and Its Applications 21. Cambridge: Cambridge Univ. Press.
• [106] Ventcel, A.D. and Freĭdlin, M.I. (1970). Small random perturbations of dynamical systems. Uspekhi Mat. Nauk 25 3–55.
• [107] Wilson, D.B. (1996). Generating random spanning trees more quickly than the cover time. In Proceedings of the Twenty-Eighth Annual ACM Symposium on the Theory of Computing (Philadelphia, PA, 1996) 296–303. New York: ACM.
• [108] Wu, F.Y. (2002). Dimers and spanning trees: Some recent results. Internat. J. Modern Phys. B 16 1951–1961.
• [109] Zeilberger, D. (1985). A combinatorial approach to matrix algebra. Discrete Math. 56 61–72.
• [110] Zhang, Y., Yong, X. and Golin, M.J. (2005). Chebyshev polynomials and spanning tree formulas for circulant and related graphs. Discrete Math. 298 334–364.