The Annals of Statistics

Continuous Time Control of Markov Processes on an Arbitrary State Space: Discounted Rewards

Bharat T. Doshi

Full-text: Open access

Abstract

The paper deals with continuous time Markov decision processes on a fairly general state space. The rewards are continuously discounted at rate $\alpha > 0$. A set of conditions is shown to be necessary and sufficient for a policy to be optimal. For the special case of time independent reward function and under the assumption that the action space is finite a policy improvement algorithm is proposed and its convergence to an optimal policy is proved.

Article information

Source
Ann. Statist., Volume 4, Number 6 (1976), 1219-1235.

Dates
First available in Project Euclid: 12 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aos/1176343653

Digital Object Identifier
doi:10.1214/aos/1176343653

Mathematical Reviews number (MathSciNet)
MR444249

Zentralblatt MATH identifier
0345.93073

JSTOR
links.jstor.org

Subjects
Primary: 49C15
Secondary: 60K99: None of the above, but in this section

Keywords
Continuous time Markov decision processes optimal discounted return function optimal policy

Citation

Doshi, Bharat T. Continuous Time Control of Markov Processes on an Arbitrary State Space: Discounted Rewards. Ann. Statist. 4 (1976), no. 6, 1219--1235. doi:10.1214/aos/1176343653. https://projecteuclid.org/euclid.aos/1176343653


Export citation