Journal of Applied Mathematics

A Version of the Euler Equation in Discounted Markov Decision Processes

H. Cruz-Suárez, G. Zacarías-Espinoza, and V. Vázquez-Guevara

Full-text: Open access

Abstract

This paper deals with Markov decision processes (MDPs) on Euclidean spaces with an infinite horizon. An approach to study this kind of MDPs is using the dynamic programming technique (DP). Then the optimal value function is characterized through the value iteration functions. The paper provides conditions that guarantee the convergence of maximizers of the value iteration functions to the optimal policy. Then, using the Euler equation and an envelope formula, the optimal solution of the optimal control problem is obtained. Finally, this theory is applied to a linear-quadratic control problem in order to find its optimal policy.

Article information

Source
J. Appl. Math., Volume 2012 (2012), Article ID 103698, 16 pages.

Dates
First available in Project Euclid: 2 January 2013

Permanent link to this document
https://projecteuclid.org/euclid.jam/1357153560

Digital Object Identifier
doi:10.1155/2012/103698

Mathematical Reviews number (MathSciNet)
MR2991589

Zentralblatt MATH identifier
1272.49045

Citation

Cruz-Suárez, H.; Zacarías-Espinoza, G.; Vázquez-Guevara, V. A Version of the Euler Equation in Discounted Markov Decision Processes. J. Appl. Math. 2012 (2012), Article ID 103698, 16 pages. doi:10.1155/2012/103698. https://projecteuclid.org/euclid.jam/1357153560


Export citation