The Annals of Mathematical Statistics

Markovian Decision Processes with Compact Action Spaces

Nagata Furukawa

Full-text: Open access

Abstract

We consider the problem of maximizing the expectation of the discounted total reward in Markovian decision processes with arbitrary state space and compact action space varying with the state. We get the existence theorem for a $(p, \epsilon)$-optimal stationary policy, and the relation between the optimality of a policy and the optimality equation. Assuming the action space is a compact subset of $n$-dimensional Euclidean space, the existence of an optimal stationary policy is established, and an algorithm is obtained for finding the optimal policy. The last two facts are based on the Borel implicit function lemma given in this paper.

Article information

Source
Ann. Math. Statist., Volume 43, Number 5 (1972), 1612-1622.

Dates
First available in Project Euclid: 27 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aoms/1177692393

Digital Object Identifier
doi:10.1214/aoms/1177692393

Mathematical Reviews number (MathSciNet)
MR371418

Zentralblatt MATH identifier
0277.90083

JSTOR
links.jstor.org

Citation

Furukawa, Nagata. Markovian Decision Processes with Compact Action Spaces. Ann. Math. Statist. 43 (1972), no. 5, 1612--1622. doi:10.1214/aoms/1177692393. https://projecteuclid.org/euclid.aoms/1177692393


Export citation