Stochastic Systems

Construction of asymptotically optimal control for crisscross network from a free boundary problem

Amarjit Budhiraja, Xin Liu, and Subhamay Saha

Full-text: Open access


An asymptotic framework for optimal control of multiclass stochastic processing networks, using formal diffusion approximations under suitable temporal and spatial scaling, by Brownian control problems (BCP) and their equivalent workload formulations (EWF), has been developed by Harrison (1988). This framework has been implemented in many works for constructing asymptotically optimal control policies for a broad range of stochastic network models. To date all asymptotic optimality results for such networks correspond to settings where the solution of the EWF is a reflected Brownian motion in $\mathbb{R} _{+}$ or a wedge in $\mathbb{R} _{+}^{2}$. In this work we consider a well studied stochastic network which is perhaps the simplest example of a model with more than one dimensional workload process. In the regime considered here, the singular control problem corresponding to the EWF does not have a simple form explicit solution. However, by considering an associated free boundary problem one can give a representation for an optimal controlled process as a two dimensional reflected Brownian motion in a Lipschitz domain whose boundary is determined by the solution of the free boundary problem. Using the form of the optimal solution we propose a sequence of control policies, given in terms of suitable thresholds, for the scaled stochastic network control problems and prove that this sequence of policies is asymptotically optimal. As suggested by the solution of the EWF, the policy we propose requires a server to idle under certain conditions which are specified in terms of thresholds determined from the free boundary.

Article information

Stoch. Syst., Volume 6, Number 2 (2016), 459-518.

Received: November 2015
First available in Project Euclid: 22 March 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K25: Queueing theory [See also 68M20, 90B22] 68M20: Performance evaluation; queueing; scheduling [See also 60K25, 90Bxx] 90B36: Scheduling theory, stochastic [See also 68M20]
Secondary: 60J70: Applications of Brownian motions and diffusion theory (population genetics, absorption problems, etc.) [See also 92Dxx]

Stochastic networks crisscross networks dynamic control heavy traffic diffusion approximations Brownian control problems singular control problems reflected Brownian motions free boundary problems threshold policies large deviations

Creative Commons Attribution 4.0 International License.


Budhiraja, Amarjit; Liu, Xin; Saha, Subhamay. Construction of asymptotically optimal control for crisscross network from a free boundary problem. Stoch. Syst. 6 (2016), no. 2, 459--518. doi:10.1214/15-SSY211.

Export citation


  • [1] S. L. Bell and R. J. Williams. Dynamic scheduling of a system with two parallel servers in heavy traffic with resource pooling: Asymptotic optimality of a threshold policy. Ann. Appl. Probab., 11(3):608–649, 2001.
  • [2] S. L. Bell and R. J. Williams. Dynamic scheduling of a parallel server system in heavy traffic with complete resource pooling: Asymptotic optimality of a threshold policy. Electron. J. Probab., 10:1044–1115, 2005.
  • [3] V. E. Beneš, L. A. Shepp, and H. S. Witsenhausen. Some solvable stochastic control problems. Stochastics, 4(1):39–83, 1980/81.
  • [4] A. Budhiraja and A. P. Ghosh. A large deviation approach to asymptotically optimal control of crisscross network in heavy traffic. The Annals of Applied Probability, 15(3):1887–1935, 2005.
  • [5] A. Budhiraja and A. P. Ghosh. Diffusion approximations for controlled stochastic networks: An asymptotic bound for the value function. Ann. Appl Probab, 16(4):1962–2006, 2006.
  • [6] A. Budhiraja and K. Ross. Convergent numerical scheme for singular stochastic control with state constraints in a portfolio selection problem. SIAM J. Control Optim., 45(6):2169–2206, 2007.
  • [7] A. Budhiraja and K. Ross. Optimal stopping and free boundary characterizations for some brownian control problems. Ann Appl. Probab., 18:2367–2391, 2008.
  • [8] A. Budhiraja and A. P. Ghosh. Controlled stochastic networks in heavy traffic: convergence of value functions. Ann. Appl. Probab., 22(2):734–791, 2012.
  • [9] A. Budhiraja, A. P. Ghosh, and X. Liu, Scheduling control for Markov modulated single-server multiclass queueing systems in heavy traffic, Queueing Systems, 78(1), 57–97, 2014.
  • [10] H. Chen and A. Mandelbeaum. Leontief systems, RBV’s and RBM’s. In M. H. A. Davis and R. J. Elliott, editors, Applied Stochastic Analysis, pages 1–43. Gordon and Breach, 1991.
  • [11] J. G. Dai and W. Lin. Asymptotic optimality of maximum pressure policies in stochastic processing networks. Ann. Appl. Probab., 18(6):2239–2299, 2008.
  • [12] S. N. Ethier and T. G. Kurtz, Markov processes: Characterization and convergence, Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley & Sons, Inc., New York, 1986.
  • [13] J. M. Harrison. Brownian models of queueing networks with heterogeneous customer population. In W. Fleming and F. L. Lion, editors, Stochastic Differential Systems, Stochastic Control Theory and Applications, pages 147–186. Springer, New York, 1988.
  • [14] J. M. Harrison. Heavy traffic analysis of a system with parallel servers: asymptotic optimality of discrete-review policies. Ann. Appl. Probab., 8(3):822–848, 1998.
  • [15] J. M. Harrison and M. I. Taksar. Instantaneous control of Brownian motion. Math. Oper. Res., 8(3):439–453, 1983.
  • [16] J. M. Harrison and J. A. Van Mieghem. Dynamic control of Brownian networks: state space collapse and equivalent workload formulations. Ann. Appl. Probab., 7(3):747–771, 1997.
  • [17] J. M. Harrison and L. M. Wein. Scheduling networks of queues: heavy traffic analysis of a simple open network. Queueing Systems Theory Appl., 5(4):265–279, 1989.
  • [18] Sunil Kumar. Two-server closed networks in heavy traffic: diffusion limits and asymptotic optimality. Ann. Appl. Probab., 10(3):930–961, 2000.
  • [19] Sunil Kumar and Kumar Muthuraman. A numerical method for solving singular stochastic control problems. Oper. Res., 52(4):563–582, 2004.
  • [20] H. J. Kushner and L. F. Martins. Numerical methods for stochastic singular control problems. SIAM J. Control Optim., 29:1443–1475, 1991.
  • [21] H. J. Kushner and L. F. Martins. Heavy traffic analysis of a controlled multiclass queueing network via weak convergence methods. SIAM J. Control Optim., 34(5):1781–1797, 1996.
  • [22] L. F. Martins, S. E. Shreve, and H. M. Soner. Heavy traffic convergence of a controlled, multiclass queueing system. SIAM J. Control Optim., 34:2133–2171, 1996.
  • [23] Kumar Muthuraman and Sunil Kumar. Solving free-boundary problems with applications in finance. Found. Trends Stoch. Syst., 1(4):259–341, 2006.
  • [24] V. Pesic and R. J. Williams, Dynamic scheduling for parallel server systems in heavy traffic: Graphical structure, decoupled workload matrix and some sufficient conditions for solvability of the Brownian control problem, Preprint.
  • [25] S. E. Shreve and H. M. Soner. A free boundary problem related to singular stochastic control. In Applied stochastic analysis (London, 1989), volume 5 of Stochastics Monogr., pages 265–301. Gordon and Breach, New York, 1991.
  • [26] A. V. Skorohod. Stochastic equations for diffusions in a bounded region. Theory Probab. Appl., (6):264–274, 1961.
  • [27] H. Mete Soner and S. E. Shreve. Regularity of the value function for a two-dimensional singular stochastic control problem. SIAM J. Control Optim., 27(4):876–907, 1989.
  • [28] P. Yang, H. Chen, and D. Yao. Control and scheduling in a two-station queueing network. Queueing Syst. Theory Appl., 18:301–332, 1994.