The Annals of Applied Probability

Systems with large flexible server pools: Instability of “natural” load balancing

Alexander L. Stolyar and Elena Yudovina

Full-text: Open access


We consider general large-scale service systems with multiple customer classes and multiple server (agent) pools, mean service times depend both on the customer class and server pool. It is assumed that the allowed activities (routing choices) form a tree (in the graph with vertices being both customer classes and server pools). We study the behavior of the system under a natural (load balancing) routing/scheduling rule, Longest-Queue Freest-Server (LQFS-LB), in the many-server asymptotic regime, such that the exogenous arrival rates of the customer classes, as well as the number of agents in each pool, grow to infinity in proportion to some scaling parameter $r$. Equilibrium point of the system under LQBS-LB is the desired operating point, with server pool loads minimized and perfectly balanced.

Our main results are as follows. (a) We show that, quite surprisingly (given the tree assumption), for certain parameter ranges, the fluid limit of the system may be unstable in the vicinity of the equilibrium point; such instability may occur if the activity graph is not “too small.” (b) Using (a), we demonstrate that the sequence of stationary distributions of diffusion-scaled processes [measuring $O(\sqrt{r})$ deviations from the equilibrium point] may be nontight, and in fact may escape to infinity. (c) In one special case of interest, however, we show that the sequence of stationary distributions of diffusion-scaled processes is tight, and the limit of stationary distributions is the stationary distribution of the limiting diffusion process.

Article information

Ann. Appl. Probab., Volume 23, Number 5 (2013), 2099-2138.

First available in Project Euclid: 28 August 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K25: Queueing theory [See also 68M20, 90B22] 60F17: Functional limit theorems; invariance principles

Many server models fluid limit diffusion limit load balancing instability tightness of invariant distributions


Stolyar, Alexander L.; Yudovina, Elena. Systems with large flexible server pools: Instability of “natural” load balancing. Ann. Appl. Probab. 23 (2013), no. 5, 2099--2138. doi:10.1214/12-AAP895.

Export citation


  • [1] Armony, M. and Ward, A. R. (2010). Fair dynamic routing in large-scale heterogeneous-server systems. Oper. Res. 58 624–637. Supplementary data available online.
  • [2] Atar, R., Shaki, Y. and Shwartz, A. (2011). A blind policy for equalizing cumulative idleness. Queueing Syst. 67 275–293.
  • [3] Gamarnik, D. and Momčilović, P. (2008). Steady-state analysis of a multiserver queue in the Halfin–Whitt regime. Adv. in Appl. Probab. 40 548–577.
  • [4] Gamarnik, D. and Stolyar, A. L. (2012). Stationary distribution of multiclass multi-server queueing system: Exponential bounds in the Halfin–Whitt regime. Queueing Syst. 71 25–51.
  • [5] Gamarnik, D. and Zeevi, A. (2006). Validity of heavy traffic steady-state approximation in generalized Jackson networks. Ann. Appl. Probab. 16 56–90.
  • [6] Gurvich, I. and Whitt, W. (2009). Queue-and-idleness-ratio controls in many-server service systems. Math. Oper. Res. 34 363–396.
  • [7] Karatzas, I. and Shreve, S. E. (1996). Brownian Motion and Stochastic Calculus, 2nd ed. Springer, New York.
  • [8] Liptser, R. S. and Shiryayev, A. N. (1989). Theory of Martingales. Mathematics and Its Applications (Soviet Series) 49. Kluwer Academic, Dordrecht. Translated from the Russian by K. Dzjaparidze [Kacha Dzhaparidze].
  • [9] Meyer, C. (2000). Matrix Analysis and Applied Linear Algebra. SIAM, Philadelphia, PA.
  • [10] Pontryagin, L. S. (1962). Ordinary Differential Equations. Elsevier, Amsterdam.
  • [11] Stolyar, A. L. and Tezcan, T. (2010). Control of systems with flexible multi-server pools: A shadow routing approach. Queueing Syst. 66 1–51.
  • [12] Stolyar, A. L. and Tezcan, T. (2011). Shadow-routing based control of flexible multiserver pools in overload. Oper. Res. 59 1427–1444.
  • [13] Supporting computations.