Open Access
August 2007 Optimal strategies for a class of sequential control problems with precedence relations
Hock Peng Chan, Cheng-Der Fuh, Inchi Hu
Ann. Statist. 35(4): 1722-1748 (August 2007). DOI: 10.1214/009053606000001569


Consider the following multi-phase project management problem. Each project is divided into several phases. All projects enter the next phase at the same point chosen by the decision maker based on observations up to that point. Within each phase, one can pursue the projects in any order. When pursuing the project with one unit of resource, the project state changes according to a Markov chain. The probability distribution of the Markov chain is known up to an unknown parameter. When pursued, the project generates a random reward depending on the phase and the state of the project and the unknown parameter. The decision maker faces two problems: (a) how to allocate resources to projects within each phase, and (b) when to enter the next phase, so that the total expected reward is as large as possible. In this paper we formulate the preceding problem as a stochastic scheduling problem and propose asymptotic optimal strategies, which minimize the shortfall from perfect information payoff. Concrete examples are given to illustrate our method.


Download Citation

Hock Peng Chan. Cheng-Der Fuh. Inchi Hu. "Optimal strategies for a class of sequential control problems with precedence relations." Ann. Statist. 35 (4) 1722 - 1748, August 2007.


Published: August 2007
First available in Project Euclid: 29 August 2007

zbMATH: 1209.62178
MathSciNet: MR2351103
Digital Object Identifier: 10.1214/009053606000001569

Primary: 62L05
Secondary: 62N99

Keywords: Kullback–Leibler number , likelihood ratio , Markov chains , Multi-armed bandits , Optimal stopping , scheduling , single-machine job sequencing , Wald’s equation

Rights: Copyright © 2007 Institute of Mathematical Statistics

Vol.35 • No. 4 • August 2007
Back to Top