## Journal of Applied Probability

### Additive functionals for discrete-time Markov chains with applications to birth-death processes

Yuanyuan Liu

#### Abstract

In this paper we are interested in bounding or calculating the additive functionals of the first return time on a set for discrete-time Markov chains on a countable state space, which is motivated by investigating ergodic theory and central limit theorems. To do so, we introduce the theory of the minimal nonnegative solution. This theory combined with some other techniques is proved useful for investigating the additive functionals. This method is used to study the functionals for discrete-time birth-death processes, and the polynomial convergence and a central limit theorem are derived.

#### Article information

Source
J. Appl. Probab., Volume 48, Number 4 (2011), 925-937.

Dates
First available in Project Euclid: 16 December 2011

https://projecteuclid.org/euclid.jap/1324046010

Digital Object Identifier
doi:10.1239/jap/1324046010

Mathematical Reviews number (MathSciNet)
MR2896659

Zentralblatt MATH identifier
1231.60080

#### Citation

Liu, Yuanyuan. Additive functionals for discrete-time Markov chains with applications to birth-death processes. J. Appl. Probab. 48 (2011), no. 4, 925--937. doi:10.1239/jap/1324046010. https://projecteuclid.org/euclid.jap/1324046010

#### References

• Asmussen, S. and Bladt, M. (1994). Poisson's equation for queues driven by a Markovian marked point process. Queueing Systems 17, 235–274.
• Chen, M.-F. (2003). From Markov Chains to Non-Equilibrium Particle Systems, 2nd edn. World Scientific, Singapore.
• Chung, K. L. (1967). Markov Chains with Stationary Transition Probabilities, 2nd edn. Springer, New York.
• Douc, R., Fort, G., Moulines, E. and Soulier, P. (2004). Practical drift conditions for subgeometric rates of convergence. Ann. Appl. Prob. 14, 1353–1377.
• Fort, G. and Moulines, E. (2003). Polynomial ergodicity of Markov transition kernels. Stoch. Process. Appl. 103, 57–99.
• Glynn, P. W. (1994). Poisson's equation for the recurrent M/G/1 queue. Adv. Appl. Prob. 26, 1044–1062.
• Grassmann, W. K. (1987). The asymptotic variance of a time average in a birth-death process. Ann. Operat. Res. 8, 165–174.
• Hou, Z. T. and Guo, Q. F. (1988). Homogeneous Denumerable Markov Processes. Springer, Berlin.
• Jarner, S. F. and Roberts, G. O. (2002). Polynomial convergence rates of Markov chains. Ann. Appl. Prob. 12, 224–247.
• Jones, G. L. (2004). On the Markov chain central limit theorem. Prob. Surveys 1, 299–320.
• Liu, Y. (2010). Augmented truncation of discrete-time Markov chains. Operat. Res. Lett. 38, 218–222.
• Liu, Y., Zhang, H. and Zhao, Y. (2010). Subgeometric ergodicity for continuous-time Markov chains. J. Math. Anal. Appl. 368, 178–189.
• Mao, Y. (2002). Ergodicity for discrete-time random walks. Beijing Shifan Daxue Xuebao 38, 729–733.
• Mao, Y. (2003). Algebraic convergence for discrete-time ergodic Markov chains. Sci. China Ser. A 46, 621–630.
• Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.
• Nummelin, E. and Tuominen, P. (1982). Geometric ergodicity of Harris recurrent Markov chains with applications to renewal theory. Stoch. Process. Appl. 12, 187–202.
• Roberts, G. O. and Rosenthal, J. S. (2004). General state space Markov chains and MCMC algorithms. Prob. Surveys 1, 20–71.
• Tuominen, P. and Tweedie, R. L. (1994). Subgeometric rates of convergence of $f$-ergodic Markov chains. Adv. Appl. Prob. 26, 775–798.
• Tweedie, R. L. (1983). Criteria for rates of convergence of Markov chains. In Probability Statistics and Analysis (London Math. Soc. Lecture Notes Ser. 79), eds J. F. C. Kingman and G. E. H.Reuter, Cambridge University Press, pp. 260–276.
• Van Doorn, E. A. and Schrijner, P. (1995). Geometric ergodicity and quasi-stationarity in discrete-time birth-death processes. J. Austral. Math. Soc. 37, 121–144.
• Whitt, W. (1992). Asymptotic formulas for Markov processes with applications to simulations. Operat. Res. 40, 279–291.