Stochastic Systems

Solving the drift control problem

Melda Ormeci Matoglu, John Vande Vate, and Huizhu Wang

Full-text: Open access

Abstract

We model the problem of managing capacity in a build-to-order environment as a Brownian drift control problem. We formulate a structured linear program that models a practical discretization of the problem and exploit a strong relationship between relative value functions and dual solutions to develop a functional lower bound for the continuous problem from a dual solution to the discrete problem. Refining the discretization proves a functional strong duality for the continuous problem. The linear programming formulation is so badly scaled, however, that solving it is beyond the capabilities of standard solvers. By demonstrating the equivalence between strongly feasible bases and deterministic unichain policies, we combinatorialize the pivoting process and by exploiting the relationship between dual solutions and relative value functions, develop a mechanism for solving the LP without ever computing its coefficients. Finally, we exploit the relationship between relative value functions and dual solutions to develop a scheme analogous to column generation for refining the discretization so as to drive the gap between the discrete approximation and the continuous problem to zero quickly while keeping the LP small. Computational studies show our scheme is much faster than simply solving a regular discretization of the problem both in terms of finding a policy with a low average cost and in terms of providing a lower bound on the optimal average cost.

Article information

Source
Stoch. Syst., Volume 5, Number 2 (2015), 324-371.

Dates
Received: November 2012
First available in Project Euclid: 23 December 2015

Permanent link to this document
https://projecteuclid.org/euclid.ssy/1450879457

Digital Object Identifier
doi:10.1214/12-SSY087

Mathematical Reviews number (MathSciNet)
MR3442430

Zentralblatt MATH identifier
1350.60088

Subjects
Primary: 60J70: Applications of Brownian motions and diffusion theory (population genetics, absorption problems, etc.) [See also 92Dxx]
Secondary: 90C05; 90C46

Keywords
Diffusion process drift control algorithm strongly feasible basis

Citation

Ormeci Matoglu, Melda; Vande Vate, John; Wang, Huizhu. Solving the drift control problem. Stoch. Syst. 5 (2015), no. 2, 324--371. doi:10.1214/12-SSY087. https://projecteuclid.org/euclid.ssy/1450879457


Export citation

References

  • Anderson, E. J., Nash, P. and Perold, A. F. (1983). Some properties of a class of continuous linear programs. SIAM Journal on Control and Optimization 21 758–765.
  • Anderson, E. J. and Nash, P. (1989). Linear Programming in Infinite-Dimensional Spaces. John Wiley, Chichester.
  • Ata, B., Harrison, J. M. and Shepp, L. A. (2005). Drift rate control of a Brownian processing system. The Annals of Applied Probability 15 1145–1160.
  • Avram, F. and Karaesmen, F. (1996). A method for computing Double Band Policies for switching between two diffusions. Probability in Engineering Sciences 10 569–590.
  • Buie, R. N. and Abrham, J. (1973). Numerical solutions to continuous linear programming problems. Mathematical Methods of Operations Research 17 107–117.
  • Chernoff, H. and Petkau, A. J. (1978). Optimal control of a Brownian motion. SIAM Journal on Applied Mathematics 34 717–731.
  • Cunningham, W. H. (1976). A network simplex method. Mathematical Programming 11 105–116.
  • Feldman, R. M. and Valdez-Flores, C. (1996). Applied Probability and Stochastic Processes. Pws Pub Co, Boston.
  • Floudas, C. A. and Pardalos, P. M. (2001). Encyclopedia of Optimization 6. Springer.
  • Ghosh, A. P. and Weerasinghe, A. P. (2007). Optimal buffer size for a stochastic processing network in heavy traffic. Queueing Systems 55 147–159.
  • Ghosh, A. P. and Weerasinghe, A. P. (2010). Optimal buffer size and dynamic rate control for a queueing system with impatient customers in heavy traffic. Stochastic Processes and Their Applications 120 2103–2141.
  • Harrison, J. M. (1985). Brownian Motion and Stochastic Flow Systems. Wiley, New York.
  • Helmes, K. and Stockbridge, R. H. (2000). Numerical comparison of controls and verification of optimality for stochastic control problems. Journal of Optimization Theory and Applications 106 107–127.
  • Helmes, K. and Stockbridge, R. H. (2008). Determining the optimal control of singular stochastic processes using linear programming. In Markov Processes and Related Fields: A Festschrift in Honor of Thomas G. Kurtz, (S. N. Ethier, J. Feng and R. H. Stockbridge, eds.). IMS Collections 137–153. IMS, Ohio.
  • Hernández-Hernández, D. and Hernández-Lerma, O. (1994). Discounted cost Markov decision processes on Borel spaces: The linear programming formulation. J. Math. Anal. Appl 183 335–351.
  • Hernández-Hernández, D., Hernández-Lerma, O. and Taksar, M. (1996). The linear programming approach to deterministic optimal control problems. Applicationes Mathematicae 24 17–33.
  • Hernández-Lerma, O. and González-Hernández, J. (1998). Infinite linear programming and multichain markov control processes in uncountable spaces. SIAM Journal on Control and Optimization 36 313–335.
  • Hernández-Lerma, O. and Lasserre, J. B. (1998). Approximation schemes for infinite linear programs. SIAM Journal on Optimization 8 973–998.
  • Klabjan, D. and Adelman, D. (2006). Existence of optimal policies for semi-Markov decision processes using duality for infinite linear programming. SIAM Journal on Control and Optimization 44 2104–2122.
  • Kumar, S. and Muthuraman, K. (2004). A numerical method for solving singular stochastic control problems. Operations Research 52 563–582.
  • Kushner, H. J. and Dupuis, P. (2001). Numerical Methods for Stochastic Control Problems in Continuous Time, second ed. Applications of Mathematics: Stochastic Modelling and Applied Probability 24. Springer-Verlag, New York.
  • Liao, Y.-C. (1984). Switching and impulsive control of reflected diffusion. Applied Mathematical Optimization 11 153–159.
  • Manne, A. S. (1960). Linear programming and sequential decisions. Management Science 6 259–267.
  • Orlin, J. B. (1985). On the simplex algorithm for networks andgeneralized networks. Mathematical Programming Studies 24 166–178.
  • Ormeci Matoglu, M. and Vande Vate, J. (2010). A Technical note on drift control with changeover cost. www.isye.gatech.edu/~jvandeva/DriftControlNote.pdf.
  • Ormeci Matoglu, M. and Vande Vate, J. (2011). Drift contol with changeover costs. Operations Research 59 427–439.
  • Perry, D. and Bar-Lev, S. K. (1989). A control of a Brownian storage system with two switchover drifts. Stochastic Analysis and Applications 7 103–115.
  • Protter, P. (2004). Stochastic Integration and Differential Equations. Springer-Verlag Berlin.
  • Pullan, M. C. (1993). An algorithm for a class of continuous linear programs. SIAM Journal on Control and Optimization 31 1558–1577.
  • Puterman, M. L. (2005). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York.
  • Rath, J. H. (1977). The optimal policy for a controlled Brownian motion process. SIAM Journal on Applied Mathematics 32 115–125.
  • Taksar, M. I. (1997). Infinite-dimensional linear programming approach to singular stochastic control. SIAM Journal of Control and Optimization 35 604–625.
  • Tijms, H. C. (2004). A First Course in Stochastic Models. John Wiley & Sons, Ltd.
  • Veinott, A. F. (1968). Extreme points of Leontief substitution systems. Linear Algebra and Applications 1 181–194.
  • Weiss, G. (2008). A simplex based algorithm to solve separated continuous linear programs. Mathematical Programming 115 151–198.