The Annals of Statistics

Asymptotic operating characteristics of an optimal change point detection in hidden Markov models

Cheng-Der Fuh

Full-text: Open access


Let ξ0,ξ1,,ξω1 be observations from the hidden Markov model with probability distribution Pθ0, and let ξω,ξω+1, be observations from the hidden Markov model with probability distribution Pθ1. The parameters θ0 and θ1 are given, while the change point ω is unknown. The problem is to raise an alarm as soon as possible after the distribution changes from Pθ0 to Pθ1, but to avoid false alarms. Specifically, we seek a stopping rule N which allows us to observe the ξ's sequentially, such that EN is large, and subject to this constraint, sup kEk(Nk|Nk) is as small as possible. Here Ek denotes expectation under the change point k, and E denotes expectation under the hypothesis of no change whatever.

In this paper we investigate the performance of the Shiryayev–Roberts–Pollak (SRP) rule for change point detection in the dynamic system of hidden Markov models. By making use of Markov chain representation for the likelihood function, the structure of asymptotically minimax policy and of the Bayes rule, and sequential hypothesis testing theory for Markov random walks, we show that the SRP procedure is asymptotically minimax in the sense of Pollak [Ann. Statist. 13 (1985) 206–227]. Next, we present a second-order asymptotic approximation for the expected stopping time of such a stopping scheme when ω=1. Motivated by the sequential analysis in hidden Markov models, a nonlinear renewal theory for Markov random walks is also given.

Article information

Ann. Statist., Volume 32, Number 5 (2004), 2305-2339.

First available in Project Euclid: 27 October 2004

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60B15: Probability measures on groups or semigroups, Fourier transforms, factorization
Secondary: 60F05: Central limit and other weak theorems 60K15: Markov renewal processes, semi-Markov processes

Asymptotic optimality change point detection first passage time limit of Bayes rules products of random matrices nonlinear Markov renewal theory Shiryayev–Roberts–Pollak procedure


Fuh, Cheng-Der. Asymptotic operating characteristics of an optimal change point detection in hidden Markov models. Ann. Statist. 32 (2004), no. 5, 2305--2339. doi:10.1214/009053604000000580.

Export citation


  • Alsmeyer, G. (1994). On the Markov renewal theorem. Stochastic Process. Appl. 50 37--56.
  • Alsmeyer, G. (2003). On the Harris recurrence of iterated random Lipschitz functions and related convergence rate results. J. Theoret. Probab. 16 217--247.
  • Bansal, R. K. and Papantoni-Kazakos, P. (1986). An algorithm for detecting a change in a stochastic process. IEEE Trans. Inform. Theory 32 227--235.
  • Basseville, M. and Nikiforov, I. V. (1993). Detection of Abrupt Changes: Theory and Application. Prentice Hall, Englewood Cliffs, NJ.
  • Braun, J. V. and Müller, H. G. (1998). Statistical methods for DNA sequence segmentation. Statist. Sci. 13 142--162.
  • Fuh, C.-D. (2003). SPRT and CUSUM in hidden Markov models. Ann. Statist. 31 942--977.
  • Fuh, C.-D. (2004). Uniform Markov renewal theory and ruin probabilities in Markov random walks. Ann. Appl. Probab. 14 1202--1241.
  • Fuh, C.-D. and Lai, T. L. (1998). Wald's equations, first passage times and moments of ladder variables in Markov random walks. J. Appl. Probab. 35 566--580.
  • Fuh, C.-D. and Lai, T. L. (2001). Asymptotic expansions in multidimensional Markov renewal theory and first passage times for Markov random walks. Adv. in Appl. Probab. 33 652--673.
  • Fuh, C.-D. and Zhang, C.-H. (2000). Poisson equation, maximal inequalities and $r$-quick convergence for Markov random walks. Stochastic Process. Appl. 87 53--67.
  • Götze, F. and Hipp, C. (1983). Asymptotic expansions for sums of weakly dependent random vectors. Z. Wahrsch. Verw. Gebiete 64 211--239.
  • Kesten, H. (1973). Random difference equations and renewal theory for products of random matrices. Acta Math. 131 207--248.
  • Kesten, H. (1974). Renewal theory for functionals of a Markov chain with general state space. Ann. Probab. 2 355--386.
  • Lai, T. L. (1995). Sequential change point detection in quality control and dynamical systems (with discussion). J. Roy. Statist. Soc. Ser. B 57 613--658.
  • Lai, T. L. (1998). Information bounds and quick detection of parameter changes in stochastic systems. IEEE Trans. Inform. Theory 44 2917--2929.
  • Lai, T. L. (2001). Sequential analysis: Some classical problems and new challenges (with discussion). Statist. Sinica 11 303--408.
  • Lai, T. L. and Siegmund, D. (1977). A nonlinear renewal theory with applications to sequential analysis. I. Ann. Statist. 5 946--954.
  • Lai, T. L. and Siegmund, D. (1979). A nonlinear renewal theory with applications to sequential analysis. II. Ann. Statist. 7 60--76.
  • Lorden, G. (1971). Procedures for reacting to a change in distribution. Ann. Math. Statist. 41 1897--1908.
  • Malinovskii, V. K. (1986). Asymptotic expansions in the central limit theorem for recurrent Markov renewal processes. Theory Probab. Appl. 31 523--526.
  • Melfi, V. F. (1992). Nonlinear Markov renewal theory with statistical applications. Ann. Probab. 20 753--771.
  • Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, New York.
  • Moustakides, G. V. (1986). Optimal stopping times for detecting changes in distribution. Ann. Statist. 14 1379--1387.
  • Niemi, S. and Nummelin, E. (1986). On non-singular renewal kernels with an application to a semigroup of transition kernels. Stochastic Process. Appl. 22 177--202.
  • Pollak, M. (1985). Optimal detection of a change in distribution. Ann. Statist. 13 206--227.
  • Pollak, M. (1987). Average run lengths of an optimal method of detecting a change in distribution. Ann. Statist. 15 749--779.
  • Pollak, M. and Siegmund, D. (1975). Approximations to the expected sample size of certain sequential tests. Ann. Statist. 3 1267--1282.
  • Ritov, Y. (1990). Decision theoretic optimality of the CUSUM procedure. Ann. Statist. 18 1464--1469.
  • Roberts, S. W. (1966). A comparison of some control chart procedures. Technometrics 8 411--430.
  • Shiryayev, A. N. (1963). On optimum methods in quickest detection problems. Theory Probab. Appl. 8 22--46.
  • Shiryayev, A. N. (1978). Optimum Stopping Rules. Springer, New York.
  • Siegmund, D. (1985). Sequential Analysis. Tests and Confidence Intervals. Springer, New York.
  • Woodroofe, M. (1976). A renewal theorem for curved boundaries and moments of first passage times. Ann. Probab. 4 67--80.
  • Woodroofe, M. (1977). Second order approximations for sequential point and interval estimation. Ann. Statist. 5 984--995.
  • Woodroofe, M. (1982). Nonlinear Renewal Theory in Sequential Analysis. SIAM, Philadelphia.
  • Yakir, B. (1994). Optimal detection of a change in distribution when the observations form a Markov chain with a finite state space. In Change-Point Problems (E. Carlstein, H. G. Müller and D. Siegmund, eds.) 346--358. IMS, Hayward, CA.
  • Yakir, B. (1997). A note on optimal detection of a change in distribution. Ann. Statist. 25 2117--2126.
  • Zhang, C.-H. (1988). A nonlinear renewal theory. Ann. Probab. 16 793--824.