Stochastic Systems

An Ergodic Control Problem for Many-Server Multiclass Queueing Systems with Cross-Trained Servers

Anup Biswas

Full-text: Open access


A Markovian queueing network is considered with d independent customer classes and d server pools in Halfin–Whitt regime. Class i customers has priority for service in pool i for i = 1, …, d, and may access some other pool if the pool has an idle server and all the servers in pool i are busy. We formulate an ergodic control problem where the running cost is given by a non-negative convex function with polynomial growth. We show that the limiting controlled diffusion is modelled by an action space which depends on the state variable. We provide a complete analysis for the limiting ergodic control problem and establish asymptotic convergence of the value functions for the queueing model.

Article information

Stoch. Syst., Volume 7, Number 2 (2017), 263-288.

Received: December 2015
Accepted: July 2017
First available in Project Euclid: 24 February 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 93E20: Optimal stochastic control
Secondary: 60H30: Applications of stochastic analysis (to PDE, etc.) 35J60: Nonlinear elliptic equations

multi-class Markovian queues reneging/abandonment Halfin-Whitt (QED) regime heavy-traffic long time-average control scheduling control stable Markov optimal control Hamilton-Jacobi-Bellman equation asymptotic optimality

Creative Commons Attribution 4.0 International License.


Biswas, Anup. An Ergodic Control Problem for Many-Server Multiclass Queueing Systems with Cross-Trained Servers. Stoch. Syst. 7 (2017), no. 2, 263--288. doi:10.1287/stsy.2017.0002.

Export citation


  • Aliprantis, CD., Border, KC. (2006) Infinite Dimensional Analysis: A Hitchhiker's Guide, Third ed. (Springer, Berlin).
  • Arapostathis, A., Pang, G. (2016) Ergodic diffusion control of multiclass multi-pool networks in the Halfin-Whitt regime. Ann. Appl. Probab. 26(5):3110–3153.
  • Arapostathis, A., Pang, G. (2017) Infinite horizon average optimality of the N-network queueing model in the Halfin-Whitt regime. Preprint arXiv:1602.03275.
  • Arapostathis, A., Biswas, A., Pang, G. (2015) Ergodic control of multi-class M/M/N + M queues in the Halfin-Whitt regime. Ann. Appl. Probab. 25(6):3511–3570.
  • Arapostathis, A., Borkar, VS., Ghosh, MK. (2012) Ergodic Control of Diffusion Processes. Encyclopedia of Mathematics and Its Applications, Vol. 143 (Cambridge University Press, UK, Cambridge).
  • Armony, M. (2005) Dynamic routing in large-scale service systems with heterogeneous servers. Queueing Syst. 51(3–4):287–329.
  • Atar, R. (2005) Scheduling control for queueing systems with many servers: Asymptotic optimality in heavy traffic. Ann. Appl. Probab. 15(4):2606–2650.
  • Atar, R., Biswas, A. (2014) Control of the multiclass G/G/1 queue in the moderate deviation regime. Ann. Appl. Probab. 24(5):2033–2069.
  • Atar, R., Giat, C., Shimkin, N. (2010) The /θ rule for many-server queues with abandonment. Oper. Res. 58(5):1427–1439.
  • Atar, R., Giat, C., Shimkin, N. (2011) On the asymptotic optimality of the /θ rule under ergodic cost. Queueing Syst. 67(2):127–144.
  • Atar, R., Mandelbaum, A., Reiman, MI. (2004) Scheduling a multi class queue with many exponential servers: Asymptotic optimality in heavy traffic. Ann. Appl. Probab. 14(3):1084–1134.
  • Atar, R., Mandelbaum, A., Shaikhet, G. (2009) Simplified control problems for multiclass many-server queueing systems. Math. Oper. Res. 34(4):795–812.
  • Biswas, A. (2014) Risk-sensitive control for the multiclass many-server queues in the moderate deviation regime. Math. Oper. Res. 39(3):908–929.
  • Biswas, A., Ishii, H., Saha, S., Wang, L. (2017) On viscosity solution of hjb equations with state constraints and reflection control. SIAM J. Control Optim. 55(1):365–396.
  • Budhiraja, A., Ghosh, AP., Lee, C. (2011) Ergodic rate control problem for single class queueing networks. SIAM J. Control Optim. 49(4):1570–1606. MR2817491 (2012e:60231).
  • Dai, JG., Tezcan, T. (2008) Optimal control of parallel server systems with many servers in heavy traffic. Queueing Syst. 59(2):95–134.
  • Dai, JG., Tezcan, T. (2011) State space collapse in many-server diffusion limits of parallel server systems. Math. Oper. Res. 36(2):271–320.
  • Gilbarg, D., Trudinger, NS. (1983) Elliptic Partial Differential Equations of Second Order. Grundlehren der Mathematischen Wissenschaften, Second ed., Vol. 224 (Springer, Berlin).
  • Gurvich, I., Whitt, W. (2009) Queue-and-idleness-ratio controls in many-server service systems. Math. Oper. Res. 34(2):363–396.
  • Harrison, JM., Zeevi, A. (2004) Dynamic scheduling of a multiclass queue in the Halfin-Whitt heavy traffic regime. Oper. Res. 52(2):243–257.
  • Ichihara, N., Sheu, S-J. (2013) Large time behavior of solutions of Hamilton-Jacobi-Bellman equations with quadratic nonlinearity in gradients. SIAM J. Math. Anal. 45(1):279–306.
  • Iravani, SMR., Kolfal, B., Oyen, MPV. (2007) Call-center labor cross-training: It’s a small world after all. Management Sci. 53(7):1102–1112.
  • Kallenberg, O. (2002) Foundations of Modern Probability. Probability and Its Applications, Second ed. (Springer, New York).
  • Kim, J., Ward, AR. (2013) Dynamic scheduling of a GI/GI/1 + GI queue with multiple customer classes. Queueing Syst. 75(2–4):339–384.
  • Krantz, SG. (2001) Function Theory of Several Complex Variables (AMS Chelsea Publishing, Providence, RI). RI Reprint of the 1992 edition. MR1846625 (2002e:32001).
  • Mandelbaum, A., Stolyar, AL. (2004) Scheduling flexible servers with convex delay costs: Heavy-traffic optimality of the generalized -rule. Oper. Res. 52(6):836–855.
  • Pang, G., Talreja, R., Whitt, W. (2007) Martingale proofs of many-server heavy-traffic limits for Markovian queues. Probab. Surv. 4:193–267.
  • Ramanan, K., Reiman, MI. (2003) Fluid and heavy traffic diffusion limits for a generalized processor sharing model. Ann. Appl. Probab. 13(1):100–139. MR1951995 (2004b:60220).
  • Ramanan, K., Reiman, MI. (2008) The heavy traffic limit of an unbalanced generalized processor sharing model. Ann. Appl. Probab. 18(1):22–58. MR2380890 (2009b:60287).
  • Sznajder, R., Filar, JA. (1992) Some comments on a theorem of Hardy and Littlewood. J. Optim. Theory Appl. 75(1):201–208. MR1189274 (93i:90110).
  • van Mieghem, JA. (1995) Dynamic scheduling with convex delay costs: The generalized rule. Ann. Appl. Probab. 5(3):809–833.
  • Ward, AR., Armony, M. (2013) Blind fair routing in large-scale service systems with heterogeneous customers and servers. Oper. Res. 61(1):228–243. MR3042753
  • Yosida, K. (1980) Functional Analysis. Grundlehren der Mathematischen Wissenschaften, Sixth ed., Vol. 123 (Springer-Verlag, Berlin).