## The Annals of Applied Probability

### Average optimality for risk-sensitive control with general state space

Anna Jaśkiewicz

#### Abstract

This paper deals with discrete-time Markov control processes on a general state space. A long-run risk-sensitive average cost criterion is used as a performance measure. The one-step cost function is nonnegative and possibly unbounded. Using the vanishing discount factor approach, the optimality inequality and an optimal stationary strategy for the decision maker are established.

#### Article information

Source
Ann. Appl. Probab., Volume 17, Number 2 (2007), 654-675.

Dates
First available in Project Euclid: 19 March 2007

https://projecteuclid.org/euclid.aoap/1174323259

Digital Object Identifier
doi:10.1214/105051606000000790

Mathematical Reviews number (MathSciNet)
MR2308338

Zentralblatt MATH identifier
1128.93056

#### Citation

Jaśkiewicz, Anna. Average optimality for risk-sensitive control with general state space. Ann. Appl. Probab. 17 (2007), no. 2, 654--675. doi:10.1214/105051606000000790. https://projecteuclid.org/euclid.aoap/1174323259

#### References

• Balaji, S. and Meyn, S. P. (2000). Multiplicative ergodicity and large deviations for an irreducible Markov chains. Stochastic Process. Appl. 90 123--144.
• Berge, E. (1963). Topological Spaces. MacMillan, New York.
• Bielecki, T., Hernández-Hernández, D. and Pliska, S. (1999). Risk-senisitive control of finite state Markov chains in discrete time, with applications to portfolio managment. Math. Methods Oper. Res. 50 167--188.
• Bielecki, T. and Pliska, S. (1999). Risk-senisitive dynamic asset managment. Appl. Math. Optim. 39 337--360.
• Borkar, V. S. and Meyn, S. P. (2002). Risk-sensitive optimal control for Markov decision processes with monotone cost. Math. Oper. Res. 27 192--209.
• Brown, L. D. and Purves, R. (1973). Measurable selections of extrema. Ann. Statist. 1 902--912.
• Cavazos-Cadena, R. (1991). A counterexample on the optimality equation in Markov decision chains with the average cost criterion. Systems Control Lett. 16 387--392.
• Cavazos-Cadena, R. and Fernández-Gaucherand, E. (1999). Controlled Markov chains with risk-sensitive criteria: Average cost, optimal equations and optimal solutions. Math. Methods Oper. Res. 49 299--324.
• Dai Pra, P., Meneghini, L. and Runggaldier, W. J. (1996). Some connections between stochastic control and dynamic games. Math. Control Signals Systems 9 303--326.
• Di Masi, G. B. and Stettner, Ł. (2000). Risk-sensitive control of discrete-time Markov processes with infinite horizon. SIAM J. Control Optim. 38 61--78.
• Di Masi, G. B. and Stettner, Ł. (2000). Infinite horizon risk sensitive control of discrete time Markov processes with small risk. Systems Control Lett. 40 15--20.
• Dupuis, P. and Ellis, R. S. (1997). A Weak Convergence Approach to the Theory of Large Deviations. Wiley, New York.
• Filar, J. and Vrieze, K. (1997). Competitive Markov Decision Processes. Springer, New York.
• Fleming, W. H. and Hernández-Hernández, D. (1997). Risk-sensitive control of finite state machines on an infinite horizon. SIAM J. Control Optim. 35 1790--1810.
• Hernández-Hernández, D. and Marcus, S. I. (1996). Risk sensitive control of Markov processes in countable state space. Systems Control Lett. 29 147--155. [Corrigendum (1998) Systems Control Lett. 34 105--106.]
• Hernández-Hernández, D. and Marcus, S. I. (1999). Existence of risk-sensitive optimal stationary policies for controlled Markov processes. Appl. Math. Optim. 40 273--285.
• Hernández-Lerma, O. and Lasserre, J. B. (1993). Discrete-Time Markov Control Process: Basic Optimality Criteria. Springer, New York.
• Howard, R. A. and Matheson, J. E. (1972). Risk-sensitive Markov decision processes. Management Sci. 18 356--369.
• Jacobson, D. H. (1973). Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games. IEEE Trans. Automat. Control 18 124--131.
• Jaśkiewicz, A. (2006). A note on risk-sensitive control of invariant models. Technical Report, Wrocław University of Technology.
• Jaśkiewicz, A. and Nowak, A. S. (2006). On the optimality equation for average cost Markov control processes with Feller transition probabilities. J. Math. Anal. Appl. 316 495--509.
• Jaśkiewicz, A. and Nowak, A. S. (2006). Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J. Control Optim. 45 773--789.
• Klein, E. and Thompson, A. C. (1984). Theory of Correspondences. Wiley, New York.
• Neveu, J. (1965). Mathematical Foundations of the Calculus of Probability. Holden-Day, San Francisco, CA.
• Royden, H. L. (1968). Real Analysis. MacMillan, New York.
• Schäl, M. (1975). Conditions for optimality in dynamic programming and for the limit $n$-stage optimal policies to be optimal. Z. Wahrsch. Verw. Gebiete 32 179--196.
• Schäl, M. (1993). Average optimality in dynamic programming with general state space. Math. Oper. Res. 18 163--172.
• Sennott, L. I. (1999). Stochastic Dynamic Programming and the Control of Queueing Systems. Wiley, New York.
• Serfozo, R. (1982). Convergence of Lebesgue integrals with varying measures. Sankhyã Ser. A 44 380--402.
• Stettner, \L. (1999). Risk sensitive portfolio optimization. Math. Methods Oper. Res. 50 463--474.
• Whittle, P. (1990). Risk-Sensitive Optimal Control. Wiley, Chichester.