The Annals of Probability

The Optimal Reward Operator in Special Classes of Dynamic Programming Problems

David A. Freedman

Full-text: Open access

Abstract

Consider a dynamic programming problem with separable metric state space $S$, constraint set $A$, and reward function $r(x, P, y)$ for $(x, P)\in A$ and $y\in S$. Let $Tf$ be the optimal reward in one move, for the reward function $r(x, P, y) + f(y)$. Three results are proved. First, suppose $S$ is compact, $A$ closed, and $r$ upper semi-continuous; then $T^n0$ is upper semi-continuous, and there is an optimal Borel strategy for the $n$-move game. Second, suppose $S$ is compact, $A$ is an $F_\sigma$, and $\{r > a\}$ is an $F_\sigma$ for all $a$; then $\{T^n0 > a\}$ is an $F_\sigma$ for all $a$, and there is an $\varepsilon$-optimal Borel strategy for the $n$-move game. Third, suppose $A$ is open and $r$ is lower semi-continuous; then $T^n0$ is lower semi-continuous, and there is an $\varepsilon$-optimal Borel measurable strategy for the $n$-move game.

Article information

Source
Ann. Probab., Volume 2, Number 5 (1974), 942-949.

Dates
First available in Project Euclid: 19 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aop/1176996559

Digital Object Identifier
doi:10.1214/aop/1176996559

Mathematical Reviews number (MathSciNet)
MR359819

Zentralblatt MATH identifier
0318.49022

JSTOR
links.jstor.org

Subjects
Primary: 49C99
Secondary: 60K99: None of the above, but in this section 90C99: None of the above, but in this section 28A05: Classes of sets (Borel fields, $\sigma$-rings, etc.), measurable sets, Suslin sets, analytic sets [See also 03E15, 26A21, 54H05]

Keywords
Dynamic programming optimal reward optimal strategy gambling

Citation

Freedman, David A. The Optimal Reward Operator in Special Classes of Dynamic Programming Problems. Ann. Probab. 2 (1974), no. 5, 942--949. doi:10.1214/aop/1176996559. https://projecteuclid.org/euclid.aop/1176996559


Export citation