A Modified Form of the Iterative Method of Dynamic Programming

Abstract

This paper considers the discrete time finite state Markovian decision problem with the average return criterion. A modified form of the iterative method of dynamic programming is studied. Under the assumption that the maximal average return is independent of the initial state the asymptotic behaviour of the sequence of functions generated by this modified method is found. It is shown that the modified iterative method supplies both upper and lower bounds on the maximal average return and $\varepsilon$-optimal policies. Moreover, a convergence result is proved for the policies produced by the modified iterative method.