The Annals of Applied Probability

Perfect sampling for nonhomogeneous Markov chains and hidden Markov models

Nick Whiteley and Anthony Lee

Full-text: Open access


We obtain a perfect sampling characterization of weak ergodicity for backward products of finite stochastic matrices, and equivalently, simultaneous tail triviality of the corresponding nonhomogeneous Markov chains. Applying these ideas to hidden Markov models, we show how to sample exactly from the finite-dimensional conditional distributions of the signal process given infinitely many observations, using an algorithm which requires only an almost surely finite number of observations to actually be accessed. A notion of “successful” coupling is introduced and its occurrence is characterized in terms of conditional ergodicity properties of the hidden Markov model and related to the stability of nonlinear filters.

Article information

Ann. Appl. Probab., Volume 26, Number 5 (2016), 3044-3077.

Received: October 2014
Revised: December 2015
First available in Project Euclid: 19 October 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 60G35: Signal detection and filtering [See also 62M20, 93E10, 93E11, 94Axx]

Coupling conditional ergodicity nonhomogeneous Markov chains perfect simulation


Whiteley, Nick; Lee, Anthony. Perfect sampling for nonhomogeneous Markov chains and hidden Markov models. Ann. Appl. Probab. 26 (2016), no. 5, 3044--3077. doi:10.1214/15-AAP1169.

Export citation


  • Baxendale, P., Chigansky, P. and Liptser, R. (2004). Asymptotic stability of the Wonham filter: Ergodic and nonergodic signals. SIAM J. Control Optim. 43 643–669 (electronic).
  • Chigansky, P., Liptser, R. and Van Handel, R. (2011). Intrinsic methods in filter stability. In The Oxford Handbook of Nonlinear Filtering 319–351. Oxford Univ. Press, Oxford.
  • Doob, J. L. (1953). Stochastic Processes. Wiley, New York.
  • Foss, S. G. and Tweedie, R. L. (1998). Perfect simulation and backward coupling. Comm. Statist. Stochastic Models 14 187–203.
  • Glynn, P. W. and Thorisson, H. (2001). Two-sided taboo limits for Markov processes and associated perfect simulation. Stochastic Process. Appl. 91 1–20.
  • Häggström, O. (2002). Finite Markov Chains and Algorithmic Applications. Cambridge Univ. Press, Cambridge.
  • Lindvall, T. (2002). Lectures on the Coupling Method. Dover Publications, Inc., Mineola, NY.
  • Meyn, S. and Tweedie, R. L. (2009). Markov Chains and Stochastic Stability, 2nd ed. Cambridge Univ. Press, Cambridge.
  • Murdoch, D. J. and Green, P. J. (1998). Exact sampling from a continuous state space. Scand. J. Stat. 25 483–502.
  • Propp, J. G. and Wilson, D. B. (1996). Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Structures Algorithms 9 223–252.
  • Seneta, E. (2006). Non-negative Matrices and Markov Chains. Springer, New York.
  • Stenflo, Ö. (2008). Perfect sampling from the limit of deterministic products of stochastic matrices. Electron. Commun. Probab. 13 474–481.
  • van Handel, R. (2009). The stability of conditional Markov processes and Markov chains in random environments. Ann. Probab. 37 1876–1925.
  • von Weizsäcker, H. (1983). Exchanging the order of taking suprema and countable intersections of $\sigma$-algebras. Ann. Inst. H. Poincaré Sect. B (N.S.) 19 91–100.