The Annals of Applied Probability

Controlled stochastic networks in heavy traffic: Convergence of value functions

Amarjit Budhiraja and Arka P. Ghosh

Full-text: Open access

Enhanced PDF (607 KB)

Abstract

Scheduling control problems for a family of unitary networks under heavy traffic with general interarrival and service times, probabilistic routing and an infinite horizon discounted linear holding cost are studied. Diffusion control problems, that have been proposed as approximate models for the study of these critically loaded controlled stochastic networks, can be regarded as formal scaling limits of such stochastic systems. However, to date, a rigorous limit theory that justifies the use of such approximations for a general family of controlled networks has been lacking. It is shown that, under broad conditions, the value function of the suitably scaled network control problem converges to that of the associated diffusion control problem. This scaling limit result, in addition to giving a precise mathematical basis for the above approximation approach, suggests a general strategy for constructing near optimal controls for the physical stochastic networks by solving the associated diffusion control problem.

Article information

Source
Ann. Appl. Probab., Volume 22, Number 2 (2012), 734-791.

Dates
First available in Project Euclid: 2 April 2012

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1333372012

Digital Object Identifier
doi:10.1214/11-AAP784

Mathematical Reviews number (MathSciNet)
MR2953568

Zentralblatt MATH identifier
1244.60091

Subjects
Primary: 60K25: Queueing theory [See also 68M20, 90B22] 68M20: Performance evaluation; queueing; scheduling [See also 60K25, 90Bxx] 90B22: Queues and service [See also 60K25, 68M20] 90B36: Scheduling theory, stochastic [See also 68M20]
Secondary: 60J70: Applications of Brownian motions and diffusion theory (population genetics, absorption problems, etc.) [See also 92Dxx]

Keywords
Heavy traffic stochastic control scaling limits diffusion approximations unitary networks controlled stochastic processing networks asymptotic optimality singular control with state constraints Brownian control problem (BCP)

Citation

Budhiraja, Amarjit; Ghosh, Arka P. Controlled stochastic networks in heavy traffic: Convergence of value functions. Ann. Appl. Probab. 22 (2012), no. 2, 734--791. doi:10.1214/11-AAP784. https://projecteuclid.org/euclid.aoap/1333372012


Export citation

References

  • [1] Ata, B. and Kumar, S. (2005). Heavy traffic analysis of open processing networks with complete resource pooling: Asymptotic optimality of discrete review policies. Ann. Appl. Probab. 15 331–391.
    Zentralblatt MATH: 1071.60081
    Digital Object Identifier: doi:10.1214/105051604000000495
    Project Euclid: euclid.aoap/1106922331
  • [2] Atar, R. and Budhiraja, A. (2006). Singular control with state constraints on unbounded domain. Ann. Probab. 34 1864–1909.
    Mathematical Reviews (MathSciNet): MR2271486
    Zentralblatt MATH: 1106.93054
    Digital Object Identifier: doi:10.1214/009117906000000359
    Project Euclid: euclid.aop/1163517228
  • [3] Bell, S. L. and Williams, R. J. (2001). Dynamic scheduling of a system with two parallel servers in heavy traffic with resource pooling: Asymptotic optimality of a threshold policy. Ann. Appl. Probab. 11 608–649.
    Zentralblatt MATH: 1015.60080
    Digital Object Identifier: doi:10.1214/aoap/1015345343
    Project Euclid: euclid.aoap/1015345343
  • [4] Bell, S. L. and Williams, R. J. (2005). Dynamic scheduling of a parallel server system in heavy traffic with complete resource pooling: Asymptotic optimality of a threshold policy. Electron. J. Probab. 10 1044–1115.
    Zentralblatt MATH: 1109.60075
    Digital Object Identifier: doi:10.1214/EJP.v10-281
  • [5] Böhm, V. (1975). On the continuity of the optimal policy set for linear programs. SIAM J. Appl. Math. 28 303–306.
  • [6] Bramson, M. and Williams, R. J. (2000). On dynamic scheduling of stochastic networks in heavy traffic and some new results for the workload process. In Proceedings of the 39th IEEE Conference on Decision and Control 516–521. IEEE, Piscataway, NJ.
  • [7] Bramson, M. and Williams, R. J. (2003). Two workload properties for Brownian networks. Queueing Syst. 45 191–221.
    Zentralblatt MATH: 1131.60305
    Digital Object Identifier: doi:10.1023/A:1027372517452
  • [8] Budhiraja, A. and Ghosh, A. P. (2005). A large deviations approach to asymptotically optimal control of crisscross network in heavy traffic. Ann. Appl. Probab. 15 1887–1935.
    Zentralblatt MATH: 1080.60084
    Digital Object Identifier: doi:10.1214/105051605000000250
    Project Euclid: euclid.aoap/1121433772
  • [9] Budhiraja, A. and Ghosh, A. P. (2006). Diffusion approximations for controlled stochastic networks: An asymptotic bound for the value function. Ann. Appl. Probab. 16 1962–2006.
    Zentralblatt MATH: 1125.60096
    Digital Object Identifier: doi:10.1214/105051606000000457
    Project Euclid: euclid.aoap/1169065213
  • [10] Budhiraja, A. and Ross, K. (2006). Existence of optimal controls for singular control problems with state constraints. Ann. Appl. Probab. 16 2235–2255.
    Zentralblatt MATH: 1118.49008
    Digital Object Identifier: doi:10.1214/105051606000000556
    Project Euclid: euclid.aoap/1169065223
  • [11] Chen, H. and Yao, D. D. (2001). Fundamentals of Queueing Networks: Performance, Asymptotics, and Optimization. Applications of Mathematics (New York) 46. Springer, New York.
    Zentralblatt MATH: 0992.60003
  • [12] Dai, J. G. and Lin, W. (2008). Asymptotic optimality of maximum pressure policies in stochastic processing networks. Ann. Appl. Probab. 18 2239–2299.
    Zentralblatt MATH: 1175.90083
    Digital Object Identifier: doi:10.1214/08-AAP522
    Project Euclid: euclid.aoap/1227708918
  • [13] Dupuis, P. and Ishii, H. (1991). On Lipschitz continuity of the solution mapping to the Skorokhod problem, with applications. Stochastics Stochastics Rep. 35 31–62.
    Mathematical Reviews (MathSciNet): MR1110990
    Zentralblatt MATH: 0721.60062
    Digital Object Identifier: doi:10.1080/17442509108833688
  • [14] Dupuis, P. and Ramanan, K. (1999). Convex duality and the Skorokhod problem. I, II. Probab. Theory Related Fields 115 153–195, 197–236.
    Zentralblatt MATH: 0944.60061
    Digital Object Identifier: doi:10.1007/s004400050269
  • [15] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence. Wiley, New York.
    Zentralblatt MATH: 0592.60049
  • [16] Harrison, J. M. (1988). Brownian models of queueing networks with heterogeneous customer populations. In Stochastic Differential Systems, Stochastic Control Theory and Applications (Minneapolis, Minn., 1986). IMA Vol. Math. Appl. 10 147–186. Springer, New York.
  • [17] Harrison, J. M. (1996). The bigstep approach to flow management in stochastic processing networks. In Stochastic Networks: Theory and Applications (F. P. Kelly, S. Zachary and I. Ziedins, eds.) 57–90. Oxford Univ. Press, Oxford.
    Zentralblatt MATH: 0860.60068
  • [18] Harrison, J. M. (2000). Brownian models of open processing networks: Canonical representation of workload. Ann. Appl. Probab. 10 75–103.
    Zentralblatt MATH: 1131.60306
    Digital Object Identifier: doi:10.1214/aoap/1019737665
    Project Euclid: euclid.aoap/1019737665
  • [19] Harrison, J. M. and Reiman, M. I. (1981). Reflected Brownian motion on an orthant. Ann. Probab. 9 302–308.
    Zentralblatt MATH: 0462.60073
    Digital Object Identifier: doi:10.1214/aop/1176994471
    Project Euclid: euclid.aop/1176994471
  • [20] Harrison, J. M. and Van Mieghem, J. A. (1997). Dynamic control of Brownian networks: State space collapse and equivalent workload formulations. Ann. Appl. Probab. 7 747–771.
    Zentralblatt MATH: 0885.60080
    Digital Object Identifier: doi:10.1214/aoap/1034801252
    Project Euclid: euclid.aoap/1034801252
  • [21] Krichagina, E. V. and Taksar, M. I. (1992). Diffusion approximation for GI/G/1 controlled queues. Queueing Syst. 12 333–367.
    Mathematical Reviews (MathSciNet): MR1200872
    Zentralblatt MATH: 0760.60087
    Digital Object Identifier: doi:10.1007/BF01158808
  • [22] Kumar, S. (1999). Scheduling open queueing networks with sufficiently flexible resources. In Proceedings of the 37th Allerton Conference. Univ. Illinois.
  • [23] Kushner, H. J. (2001). Heavy Traffic Analysis of Controlled Queueing and Communication Networks. Applications of Mathematics (New York) 47. Springer, New York.
    Zentralblatt MATH: 0988.90004
  • [24] Kushner, H. J. and Dupuis, P. G. (1992). Numerical Methods for Stochastic Control Problems in Continuous Time. Applications of Mathematics (New York) 24. Springer, New York.
    Zentralblatt MATH: 0754.65068
  • [25] Kushner, H. J. and Martins, L. F. (1996). Heavy traffic analysis of a controlled multiclass queueing network via weak convergence methods. SIAM J. Control Optim. 34 1781–1797.
    Zentralblatt MATH: 0857.90041
    Digital Object Identifier: doi:10.1137/S0363012994275592
  • [26] Meyn, S. P. (2003). Sequencing and routing in multiclass queueing networks. II. Workload relaxations. SIAM J. Control Optim. 42 178–217 (electronic).
    Mathematical Reviews (MathSciNet): MR1982741
    Zentralblatt MATH: 1061.90047
    Digital Object Identifier: doi:10.1137/S036301290138376X
  • [27] Meyn, S. P. (2007). Control Techniques for Complex Networks. Cambridge Univ. Press, Cambridge.
    Mathematical Reviews (MathSciNet): MR2372453
    Zentralblatt MATH: 1139.91002
  • [28] Ward, A. R. and Kumar, S. (2008). Asymptotically optimal admission control of a queue with impatient customers. Math. Oper. Res. 33 167–202.
    Zentralblatt MATH: 1159.90017
    Digital Object Identifier: doi:10.1287/moor.1070.0287
  • [29] Williams, R. J. (1998). An invariance principle for semimartingale reflecting Brownian motions in an orthant. Queueing Syst. 30 5–25.
    Zentralblatt MATH: 0911.90170
    Digital Object Identifier: doi:10.1023/A:1019156702875

The Institute of Mathematical Statistics

The Annals of Applied Probability

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more