Journal of Applied Probability

Exact estimation for Markov chain equilibrium expectations

Peter W. Glynn and Chang-Han Rhee


We introduce a new class of Monte Carlo methods, which we call exact estimation algorithms. Such algorithms provide unbiased estimators for equilibrium expectations associated with real-valued functionals defined on a Markov chain. We provide easily implemented algorithms for the class of positive Harris recurrent Markov chains, and for chains that are contracting on average. We further argue that exact estimation in the Markov chain setting provides a significant theoretical relaxation relative to exact simulation methods.

Article information

J. Appl. Probab., Volume 51A (2014), 377-389.

First available in Project Euclid: 2 December 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 65C05: Monte Carlo methods
Secondary: 60J05: Discrete-time Markov processes on general state spaces

Unbiased estimation Markov chain equilibrium expectation Markov chain stationary expectation exact estimation exact sampling exact simulation perfect sampling perfect simulation


Glynn, Peter W.; Rhee, Chang-Han. Exact estimation for Markov chain equilibrium expectations. J. Appl. Probab. 51A (2014), 377--389. doi:10.1239/jap/1417528487.

Export citation


  • Asmussen, S. and Glynn, P. W. (2007). Stochastic Simulation: Algorithms and Analysis (Stoch. Modelling Appl. Prob. 57). Springer, New York.
  • Asmussen, S., Glynn, P. W. and Thorisson, H. (1992). Stationarity detection in the initial transient problem. ACM Trans. Model. Comput. Simul. 2, 130–157.
  • Athreya, K. B. and Ney, P. (1978). A new approach to the limit theory of recurrent Markov chains. Trans. Amer. Math. Soc. 245, 493–501.
  • Blanchet, J. and Wallwater, A. (2014). Exact sampling for the steady-state waiting time of a heavy-tailed single server queue. Submitted.
  • Borovkov, A. A. and Foss, S. G. (1992). Stochastically recursive sequences and their generalizations. Siberian Adv. Math. 2, 16–81.
  • Chung, K. L. (2001). A Course in Probability Theory, 3rd edn. Academic Press, San Diego, CA.
  • Connor, S. B. and Kendall, W. S. (2007). Perfect simulation for a class of positive recurrent Markov chains. Ann. Appl. Prob. 17, 781–808.
  • Corcoran, J. N. and Tweedie, R. L. (2001). Perfect sampling of ergodic Harris chains. Ann. Appl. Prob. 11, 438–451.
  • Diaconis, P. and Freedman, D. (1999). Iterated random functions. SIAM Rev. 41, 45–76.
  • Ensor, K. B. and Glynn, P. W. (2000). Simulating the maximum of a random walk. J. Statist. Planning Infer. 85, 127–135.
  • Giles, M. B. (2008). Multilevel Monte Carlo path simulation. Operat. Res. 56, 607–617.
  • Glynn, P. W. and Whitt, W. (1992). The asymptotic efficiency of simulation estimators. Operat. Res. 40, 505–520.
  • Kendall, W. S. (2004). Geometric ergodicity and perfect simulation. Electron. Commun. Prob. 9, 140–151.
  • Lindvall, T. (2002). Lectures on the Coupling Method. Dover Publications, Mineola, NY.
  • McLeish, D. (2011). A general method for debiasing a Monte Carlo estimator. Monte Carlo Meth. Appl. 17, 301–315.
  • Meyn, S. and Tweedie, R. L. (2009). Markov Chains and Stochastic Stability, 2nd edn. Cambridge University Press.
  • Propp, J. G. and Wilson, D. B. (1996). Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Structures Algorithms 9, 223–252.
  • Rhee, C.-H. and Glynn, P. W. (2012). A new approach to unbiased estimation for SDE's. In Proc. 2012 Winter Simulation Conference, eds. C. Laroque et al., 7pp.
  • Rhee, C.-H. and Glynn, P. W. (2013). Unbiased estimation with square root convergence for SDE models. Submitted.
  • Rychlik, T. (1990). Unbiased nonparametric estimation of the derivative of the mean. Statist. Prob. Lett. 10, 329–333.
  • Rychlik, T. (1995). A class of unbiased kernel estimates of a probability density function. Appl. Math. 22, 485–497.
  • Thorisson, H. (2000). Coupling, Stationarity, and Regeneration. Springer, New York. \endharvreferences