The Annals of Applied Probability

Average optimality for risk-sensitive control with general state space

Anna Jaśkiewicz

Full-text: Open access


This paper deals with discrete-time Markov control processes on a general state space. A long-run risk-sensitive average cost criterion is used as a performance measure. The one-step cost function is nonnegative and possibly unbounded. Using the vanishing discount factor approach, the optimality inequality and an optimal stationary strategy for the decision maker are established.

Article information

Ann. Appl. Probab., Volume 17, Number 2 (2007), 654-675.

First available in Project Euclid: 19 March 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60J05: Discrete-time Markov processes on general state spaces 90C39: Dynamic programming [See also 49L20]
Secondary: 60A10: Probabilistic measure theory {For ergodic theory, see 28Dxx and 60Fxx}

Risk-sensitive control Borel state space average cost optimality inequality


Jaśkiewicz, Anna. Average optimality for risk-sensitive control with general state space. Ann. Appl. Probab. 17 (2007), no. 2, 654--675. doi:10.1214/105051606000000790.

Export citation


  • Balaji, S. and Meyn, S. P. (2000). Multiplicative ergodicity and large deviations for an irreducible Markov chains. Stochastic Process. Appl. 90 123--144.
  • Berge, E. (1963). Topological Spaces. MacMillan, New York.
  • Bielecki, T., Hernández-Hernández, D. and Pliska, S. (1999). Risk-senisitive control of finite state Markov chains in discrete time, with applications to portfolio managment. Math. Methods Oper. Res. 50 167--188.
  • Bielecki, T. and Pliska, S. (1999). Risk-senisitive dynamic asset managment. Appl. Math. Optim. 39 337--360.
  • Borkar, V. S. and Meyn, S. P. (2002). Risk-sensitive optimal control for Markov decision processes with monotone cost. Math. Oper. Res. 27 192--209.
  • Brown, L. D. and Purves, R. (1973). Measurable selections of extrema. Ann. Statist. 1 902--912.
  • Cavazos-Cadena, R. (1991). A counterexample on the optimality equation in Markov decision chains with the average cost criterion. Systems Control Lett. 16 387--392.
  • Cavazos-Cadena, R. and Fernández-Gaucherand, E. (1999). Controlled Markov chains with risk-sensitive criteria: Average cost, optimal equations and optimal solutions. Math. Methods Oper. Res. 49 299--324.
  • Dai Pra, P., Meneghini, L. and Runggaldier, W. J. (1996). Some connections between stochastic control and dynamic games. Math. Control Signals Systems 9 303--326.
  • Di Masi, G. B. and Stettner, Ł. (2000). Risk-sensitive control of discrete-time Markov processes with infinite horizon. SIAM J. Control Optim. 38 61--78.
  • Di Masi, G. B. and Stettner, Ł. (2000). Infinite horizon risk sensitive control of discrete time Markov processes with small risk. Systems Control Lett. 40 15--20.
  • Dupuis, P. and Ellis, R. S. (1997). A Weak Convergence Approach to the Theory of Large Deviations. Wiley, New York.
  • Filar, J. and Vrieze, K. (1997). Competitive Markov Decision Processes. Springer, New York.
  • Fleming, W. H. and Hernández-Hernández, D. (1997). Risk-sensitive control of finite state machines on an infinite horizon. SIAM J. Control Optim. 35 1790--1810.
  • Hernández-Hernández, D. and Marcus, S. I. (1996). Risk sensitive control of Markov processes in countable state space. Systems Control Lett. 29 147--155. [Corrigendum (1998) Systems Control Lett. 34 105--106.]
  • Hernández-Hernández, D. and Marcus, S. I. (1999). Existence of risk-sensitive optimal stationary policies for controlled Markov processes. Appl. Math. Optim. 40 273--285.
  • Hernández-Lerma, O. and Lasserre, J. B. (1993). Discrete-Time Markov Control Process: Basic Optimality Criteria. Springer, New York.
  • Howard, R. A. and Matheson, J. E. (1972). Risk-sensitive Markov decision processes. Management Sci. 18 356--369.
  • Jacobson, D. H. (1973). Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games. IEEE Trans. Automat. Control 18 124--131.
  • Jaśkiewicz, A. (2006). A note on risk-sensitive control of invariant models. Technical Report, Wrocław University of Technology.
  • Jaśkiewicz, A. and Nowak, A. S. (2006). On the optimality equation for average cost Markov control processes with Feller transition probabilities. J. Math. Anal. Appl. 316 495--509.
  • Jaśkiewicz, A. and Nowak, A. S. (2006). Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J. Control Optim. 45 773--789.
  • Klein, E. and Thompson, A. C. (1984). Theory of Correspondences. Wiley, New York.
  • Neveu, J. (1965). Mathematical Foundations of the Calculus of Probability. Holden-Day, San Francisco, CA.
  • Royden, H. L. (1968). Real Analysis. MacMillan, New York.
  • Schäl, M. (1975). Conditions for optimality in dynamic programming and for the limit $n$-stage optimal policies to be optimal. Z. Wahrsch. Verw. Gebiete 32 179--196.
  • Schäl, M. (1993). Average optimality in dynamic programming with general state space. Math. Oper. Res. 18 163--172.
  • Sennott, L. I. (1999). Stochastic Dynamic Programming and the Control of Queueing Systems. Wiley, New York.
  • Serfozo, R. (1982). Convergence of Lebesgue integrals with varying measures. Sankhyã Ser. A 44 380--402.
  • Stettner, \L. (1999). Risk sensitive portfolio optimization. Math. Methods Oper. Res. 50 463--474.
  • Whittle, P. (1990). Risk-Sensitive Optimal Control. Wiley, Chichester.