Journal of Applied Probability

Universality of load balancing schemes on the diffusion scale

Debankur Mukherjee, Sem C. Borst, Johan S. H. van Leeuwaarden, and Philip A. Whiting

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


We consider a system of N parallel queues with identical exponential service rates and a single dispatcher where tasks arrive as a Poisson process. When a task arrives, the dispatcher always assigns it to an idle server, if there is any, and to a server with the shortest queue among d randomly selected servers otherwise (1≤dN). This load balancing scheme subsumes the so-called join-the-idle queue policy (d=1) and the celebrated join-the-shortest queue policy (d=N) as two crucial special cases. We develop a stochastic coupling construction to obtain the diffusion limit of the queue process in the Halfin‒Whitt heavy-traffic regime, and establish that it does not depend on the value of d, implying that assigning tasks to idle servers is sufficient for diffusion level optimality.

Article information

J. Appl. Probab., Volume 53, Number 4 (2016), 1111-1124.

First available in Project Euclid: 7 December 2016

Permanent link to this document

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K25: Queueing theory [See also 68M20, 90B22] 68M20: Performance evaluation; queueing; scheduling [See also 60K25, 90Bxx]
Secondary: 60K30: Applications (congestion, allocation, storage, traffic, etc.) [See also 90Bxx] 90B18: Communication networks [See also 68M10, 94A05] 90B36: Scheduling theory, stochastic [See also 68M20]

Join-the-idle-queue policy join-the-shortest-queue policy load balancing power of two routeing sample path comparison stochastic coupling supermarket mode


Mukherjee, Debankur; Borst, Sem C.; van Leeuwaarden, Johan S. H.; Whiting, Philip A. Universality of load balancing schemes on the diffusion scale. J. Appl. Probab. 53 (2016), no. 4, 1111--1124.

Export citation


  • Badonnel, R. and Burgess, M. (2008). Dynamic pull-based load balancing for autonomic servers. In Network Operations and Management Symposium, NOMS 2008, pp. 751–754.
  • Bramson, M., Lu, Y. and Prabhakar, B. (2012). Asymptotic independence of queues under randomized load balancing. Queueing Systems 71, 247–292.
  • Ephremides, A., Varaiya, P. and Walrand, J. (1980). A simple dynamic routing problem. IEEE Trans. Automatic Control 25, 690–693.
  • Eschenfeldt, P. and Gamarnik, D. (2015). Join the shortest queue with many servers. The heavy traffic asymptotics. Preprint. Available at
  • Gupta, V., Harchol-Balter, M., Sigman, K. and Whitt, W. (2007). Analysis of join-the-shortest-queue routing for web server farms. Performance Evaluation 64, 1062–1081.
  • Halfin, S. and Whitt, W. (1981). Heavy-traffic limits for queues with many exponential servers. Operat. Res. 29, 567–588.
  • Jonckheere, M. (2006). Insensitive versus efficient dynamic load balancing in networks without blocking. Queueing Systems 54, 193–202.
  • Lu, Y. \et (2011). Join-idle-queue: a novel load balancing algorithm for dynamically scalable web services. Performance Evaluation 68, 1056–1071.
  • Mitzenmacher, M. (2001). The power of two choices in randomized load balancing. IEEE Trans. Parallel Distributed Systems 12, 1094–1104.
  • Pang, G., Talreja, R. and Whitt, W. (2007). Martingale proofs of many-server heavy-traffic limits for Markovian queues. Prob. Surveys 4, 193–267.
  • Sparaggis, P. D., Towsley, D. and Cassandras, C. G. (1994). Sample path criteria for weak majorization. Adv. Appl. Prob. 26, 155–171.
  • Stolyar, A. L. (2015). Pull-based load distribution in large-scale heterogeneous service systems. Queueing Systems 80, 341–361.
  • Towsley, D. (1995). Application of majorization to control problems in queueing systems. In Scheduling Theory and Its Applications, eds P. Chrétienne et al., John Wiley, Chichester.
  • Towsley, D., Sparaggis, P. D. and Cassandras, C. G. (1992). Optimal routing and buffer allocation for a class of finite capacity queueing systems. IEEE Trans. Automatic Control 37, 1446–1451.
  • Vvedenskaya, N. D., Dobrushin, R. L. and Karpelevich, F. I. (1996). Queueing system with selection of the shortest of two queues: an asymptotic approach. Prob. Peredachi Inf. 32, 20–34.
  • Weber, R. R. (1978). On the optimal assignment of customers to parallel servers. J. Appl. Prob. 15, 406–413.
  • Whitt, W. (1986). Deciding which queue to join: some counterexamples. Operat. Res. 34, 55–62.
  • Winston, W. (1977). Optimality of the shortest line discipline. J. Appl. Prob. 14, 181–189.