Stochastic Systems

Stability of a Markov-modulated Markov chain, with application to a wireless network governed by two protocols

Sergey Foss, Seva Shneer, and Andrey Tyurlikov

Full-text: Open access


We consider a discrete-time Markov chain $(X^{t},Y^{t})$, $t=0,1,2,\ldots$ , where the $X$-component forms a Markov chain itself. Assume that $(X^{t})$ is Harris-ergodic and consider an auxiliary Markov chain $\{\widehat{Y}^{t}\}$ whose transition probabilities are the averages of transition probabilities of the $Y$-component of the $(X,Y)$-chain, where the averaging is weighted by the stationary distribution of the $X$-component.

We first provide natural conditions in terms of test functions ensuring that the $\widehat{Y}$-chain is positive recurrent and then prove that these conditions are also sufficient for positive recurrence of the original chain $(X^{t},Y^{t})$. The we prove a “multi-dimensional” extension of the result obtained. In the second part of the paper, we apply our results to two versions of a multi-access wireless model governed by two randomised protocols.

Article information

Stoch. Syst., Volume 2, Number 1 (2012), 208-231.

First available in Project Euclid: 24 February 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 93E15: Stochastic stability
Secondary: 60K25: Queueing theory [See also 68M20, 90B22] 68M20: Performance evaluation; queueing; scheduling [See also 60K25, 90Bxx]

Stochastic stability Markov-modulated Markov chain


Foss, Sergey; Shneer, Seva; Tyurlikov, Andrey. Stability of a Markov-modulated Markov chain, with application to a wireless network governed by two protocols. Stoch. Syst. 2 (2012), no. 1, 208--231. doi:10.1214/11-SSY030.

Export citation


  • [1] Abramson, N. (1970). The ALOHA System - Another Alternative for Computer Communications. AFIPS Conference Proceedings. 36, 295–298.
  • [2] Andreev, S., Dubkov, K., Turlikov, A. (2010). IEEE 802.11 and 802.16 cooperation within multi-radio stations. Wireless Personal Communications, Springer Science+Business Media B.V., 1–19.
  • [3] Berlemann, L., Hoymann, C., Hiertz, G.R., Mangold, S. (2006). Coexistence and interworking of IEEE 802.16 and IEEE 802.11(e). IEEE 63rd Vehicular Technology Conference, 1, 27–31.
  • [4] Bonald, T., Borst, S., Hedge, N., Proutiere, A. (2004). Wireless Data Performance in Multi-Cell Scenarios. Proc. of ACM SIGMETRICS, 378–387.
  • [5] Bordenave, C., McDonald, D., Proutiere, A. (2008). Performance of Random Muedium Access Control, An Asymptotic Approach. Proc. of ACM SIGMETRICS/Performance, 1–12.
  • [6] Borovkov, A.A. (1998). Ergodicity and Stability of Stochastic Processes, Wiley.
  • [7] Borst, S., Jonckheere, M., Leskelä, L. Stability of parallel queueing systems with coupled service rates. Discrete Event Dynamic Systems, 18, 4, 447–472.
  • [8] Bramson, M. (2008). Stability of queueing networks, Probab. Surveys, 5, 169–345.
  • [9] Dai, J.G. (1995). On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limits. Annals Applied Probability, 5, 49–77.
  • [10] Dai, J.G. and Meyn, S.P. (1995). Stability and convergence of moments for multiclass queueing networks via fluid limit models. IEEE Transactions on Automatic Control., 40, 11, 1889–1904.
  • [11] Ephremides A., Hajek, B. (1998). Information Theory and Communication Networks: an Unconsummated Union. IEEE Transactions on Information Theory. 44, 2416–2434.
  • [12] Foss, S., Konstantopoulos, T. (2004). An overview of some stochastic stability methods. Journal of Operation Research Society Japan, 47, 275–303.
  • [13] Gamarnik, D. (2004). Stochastic bandwidth packing process: stability conditions via Lyapunov function technique. Queueing systems, 48, 339–363.
  • [14] Gamarnik, D., Squillante, M. (2005). Analysis of stochastic online bin packing processes. Stochastic Models, 21, 401–425.
  • [15] Kifer, Yu. (1986). Ergodic theory of random transformations. Progress in Probability and Statistics, Birkhauser.
  • [16] Lindvall, T. (2002). Lectures on the Coupling Method, 2nd edition, Dover.
  • [17] Litvak, N., Robert, P. (2012). A scaling analysis of a cat and mouse Markov chain. Annals of Probability, 22, 2, 792–826.
  • [18] Meyn, S.P. (2007). Control Techniques for Complex Networks, Cambridge University Press.
  • [19] Meyn, S.P., Tweedie, R.L. (1993). Markov Chains and Stochastic Stability, Springer Verlag.
  • [20] Mikhailov, V.A., Tsybakov, B.S. (1979). Ergodicity of a Slotted ALOHA System. Problems of Information Transmission, 15, 301–312.
  • [21] Roberts, L. (1972). ALOHA Packet System with and without Slots and Capture. Advanced research projects agency, Network information center, Tech. Rep. ASS Note 8.
  • [22] Shah, D., Shin, J. (2012). Randomized scheduling algorithm for queueing networks. Annals of Applied Probability, to appear.
  • [23] Thorisson, H. (2000). Coupling, Stationarity, and Regeneration, Springer Verlag.
  • [24] Walke, B., Mangold, S., Berlemann, L. (2007). IEEE 802 wireless systems: Protocols, multi-Hop mesh/relaying, performance and spectrum coexistence. NJ: Wiley.
  • [25] Zhu, J., Waltho, A., Yang, X., Guo, X. (2007). Multi-radio coexistence: Challenges and opportunities. Proceedings of 16th International Conference on Computer Communications and Networks, 13-16, 358–364.