The Annals of Applied Probability

Diffusion approximations for controlled weakly interacting large finite state systems with simultaneous jumps

Amarjit Budhiraja and Eric Friedlander

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


We consider a rate control problem for an $N$-particle weakly interacting finite state Markov process. The process models the state evolution of a large collection of particles and allows for multiple particles to change state simultaneously. Such models have been proposed for large communication systems (e.g., ad hoc wireless networks) but are also suitable for other settings such as chemical-reaction networks. An associated diffusion control problem is presented and we show that the value function of the $N$-particle controlled system converges to the value function of the limit diffusion control problem as $N\to\infty$. The diffusion coefficient in the limit model is typically degenerate; however, under suitable conditions there is an equivalent formulation in terms of a controlled diffusion with a uniformly nondegenerate diffusion coefficient. Using this equivalence, we show that near optimal continuous feedback controls exist for the diffusion control problem. We then construct near asymptotically optimal control policies for the $N$-particle system based on such continuous feedback controls. Results from some numerical experiments are presented.

Article information

Ann. Appl. Probab., Volume 28, Number 1 (2018), 204-249.

Received: March 2016
First available in Project Euclid: 3 March 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K35: Interacting random processes; statistical mechanics type models; percolation theory [See also 82B43, 82C43] 60H30: Applications of stochastic analysis (to PDE, etc.) 93E20: Optimal stochastic control
Secondary: 60J28: Applications of continuous-time Markov processes on discrete state spaces 60J70: Applications of Brownian motions and diffusion theory (population genetics, absorption problems, etc.) [See also 92Dxx] 60K25: Queueing theory [See also 68M20, 90B22] 91B70: Stochastic models

Mean field approximations diffusion approximations asymptotic optimality stochastic networks stochastic control propagation of chaos rate control ad hoc wireless networks


Budhiraja, Amarjit; Friedlander, Eric. Diffusion approximations for controlled weakly interacting large finite state systems with simultaneous jumps. Ann. Appl. Probab. 28 (2018), no. 1, 204--249. doi:10.1214/17-AAP1303.

Export citation


  • [1] Antunes, N., Fricker, C., Robert, P. and Tibi, D. (2008). Stochastic networks with multiple stable points. Ann. Probab. 36 255–278.
  • [2] Atar, R. (2005). A diffusion model of scheduling control in queueing systems with many servers. Ann. Appl. Probab. 15 820–852.
  • [3] Bell, S. L. and Williams, R. J. (2001). Dynamic scheduling of a system with two parallel servers in heavy traffic with resource pooling: Asymptotic optimality of a threshold policy. Ann. Appl. Probab. 11 608–649.
  • [4] Bensoussan, A. (1982). Stochastic Control by Functional Analysis Methods. Studies in Mathematics and Its Applications 11. North-Holland, Amsterdam.
  • [5] Billingsley, P. (1999). Convergence of Probability Measures, 2nd ed. Wiley, New York.
  • [6] Borkar, V. S. (1989). Optimal Control of Diffusion Processes. Pitman Research Notes in Mathematics Series 203. Longman Scientific & Technical, Harlow.
  • [7] Budhiraja, A. and Ghosh, A. P. (2012). Controlled stochastic networks in heavy traffic: Convergence of value functions. Ann. Appl. Probab. 22 734–791.
  • [8] Budhiraja, A., Ghosh, A. P. and Lee, C. (2011). Ergodic rate control problem for single class queueing networks. SIAM J. Control Optim. 49 1570–1606.
  • [9] Chen, Z. and Huan, Z. (1997). On the continuity of the $m$th root of a continuous nonnegative definite matrix-valued function. J. Math. Anal. Appl. 209 60–66.
  • [10] Dai, J. G. and Lin, W. (2008). Asymptotic optimality of maximum pressure policies in stochastic processing networks. Ann. Appl. Probab. 18 2239–2299.
  • [11] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence. Wiley, New York.
  • [12] Fleming, W. H. and Rishel, R. W. (1975). Deterministic and Stochastic Optimal Control. Applications of Mathematics 1. Springer, Berlin.
  • [13] Harrison, J. M. (1988). Brownian models of queueing networks with heterogeneous customer populations. In Stochastic Differential Systems, Stochastic Control Theory and Applications (Minneapolis, Minn., 1986). IMA Vol. Math. Appl. 10 147–186. Springer, New York.
  • [14] Joffe, A. and Métivier, M. (1986). Weak convergence of sequences of semimartingales with applications to multitype branching processes. Adv. in Appl. Probab. 18 20–65.
  • [15] Krylov, N. V. (2009). Controlled Diffusion Processes. Stochastic Modelling and Applied Probability 14. Springer, Berlin.
  • [16] Kurtz, T. G. (1970). Solutions of ordinary differential equations as limits of pure jump Markov processes. J. Appl. Probab. 7 49–58.
  • [17] Kurtz, T. G. (1971). Limit theorems for sequences of jump Markov processes approximating ordinary differential processes. J. Appl. Probab. 8 344–356.
  • [18] Kurtz, T. G. and Protter, P. (1991). Weak limit theorems for stochastic integrals and stochastic differential equations. Ann. Probab. 19 1035–1070.
  • [19] Kushner, H. J. (2001). Heavy Traffic Analysis of Controlled Queueing and Communication Networks. Applications of Mathematics (New York): Stochastic Modelling and Applied Probability 47. Springer, New York.
  • [20] Protter, P. E. (2005). Stochastic Integration and Differential Equations, 2nd ed. Stochastic Modelling and Applied Probability 21. Springer, Berlin.
  • [21] Rudin, W. (1986). Real and Complex Analysis, 3rd ed. McGraw-Hill, New York.
  • [22] Stroock, D. W. and Varadhan, S. R. S. (2006). Multidimensional Diffusion Processes. Springer, Berlin.
  • [23] Whitt, W. (2002). Stochastic-Process Limits: An Introduction to Stochastic-Process Limits and Their Application to Queues. Springer, New York.