The Annals of Probability

On the structure of solutions of ergodic type Bellman equation related to risk-sensitive control

Hidehiro Kaise and Shuenn-Jyi Sheu

Full-text: Open access

Abstract

Bellman equations of ergodic type related to risk-sensitive control are considered. We treat the case that the nonlinear term is positive quadratic form on first-order partial derivatives of solution, which includes linear exponential quadratic Gaussian control problem. In this paper we prove that the equation in general has multiple solutions. We shall specify the set of all the classical solutions and classify the solutions by a global behavior of the diffusion process associated with the given solution. The solution associated with ergodic diffusion process plays particular role. We shall also prove the uniqueness of such solution. Furthermore, the solution which gives us ergodicity is stable under perturbation of coefficients. Finally, we have a representation result for the solution corresponding to the ergodic diffusion.

Article information

Source
Ann. Probab., Volume 34, Number 1 (2006), 284-320.

Dates
First available in Project Euclid: 17 February 2006

Permanent link to this document
https://projecteuclid.org/euclid.aop/1140191539

Digital Object Identifier
doi:10.1214/009117905000000431

Mathematical Reviews number (MathSciNet)
MR2206349

Zentralblatt MATH identifier
1092.60030

Subjects
Primary: 60G35: Signal detection and filtering [See also 62M20, 93E10, 93E11, 94Axx]
Secondary: 60H30: Applications of stochastic analysis (to PDE, etc.) 93E20: Optimal stochastic control

Keywords
Ergodic type Bellman equations risk-senstive control classification of solutions transience and ergodicity variational representation

Citation

Kaise, Hidehiro; Sheu, Shuenn-Jyi. On the structure of solutions of ergodic type Bellman equation related to risk-sensitive control. Ann. Probab. 34 (2006), no. 1, 284--320. doi:10.1214/009117905000000431. https://projecteuclid.org/euclid.aop/1140191539


Export citation

References

  • Bensoussan, A. (1988). Perturbation Methods in Optimal Control. Wiley, New York.
  • Bensoussan, A. and Frehse, J. (1992). On Bellman equations of ergodic control in $\mathbbR^N$. J. Reine Angew. Math. 429 125--160.
  • Bielecki, T. R. and Pliska, S. R. (1999). Risk sensitive dynamic asset management. Appl. Math. Optim. 39 337--360.
  • Donsker, M. D. and Varadhan, S. R. S. (1975). Asymptotic evaluation of certain Wiener integrals for large time. In Functional Integration and Its Applications (A. M. Arthurs, ed.) 15--33. Oxford Univ. Press.
  • Donsker, M. D. and Varadhan, S. R. S. (1976). Asymptotic evaluation of certain Markov processes expectations for large time III. Comm. Pure Appl. Math. 29 389--461.
  • Dunford, N. and Schwartz, J. T. (1988). Linear Operators Part I: General Theory. Wiley, New York.
  • Ekeland, I. and Teman, R. (1976). Convex Analysis and Variational Problems. North-Holland, Amsterdam.
  • Fleming, W. H. (1995). Optimal investment model and risk-sensitive stochastic control. In IMA Vols. in Math. Appl. 65 75--88. Springer, New York.
  • Fleming, W. H. and James, M. R. (1995). The risk-sensitive index and the $H_2$ and $H_\infty$ norms for nonlinear systems. Math. Control Signals Systems 8 199--221.
  • Fleming, W. H. and McEneaney, W. M. (1995). Risk-sensitive control on an infinite time horizon. SIAM J. Control Optim. 33 1881--1915.
  • Fleming, W. H. and Sheu, S.-J. (1999). Optimal long term growth rate of expected utility of wealth. Ann. Appl. Probab. 9 871--903.
  • Fleming, W. H. and Sheu, S.-J. (2000). Risk sensitive control and an optimal investment model. Math. Finance 10 197--213.
  • Kaise, H. and Nagai, H. (1998). Bellman--Isaacs equations of ergodic type related to risk-sensitive control and their singular limits. Asymptotic Anal. 16 347--362.
  • Kaise, H. and Nagai, H. (1999). Ergodic type Bellman equations of risk-sensitive control with large parameters and their singular limits. Asymptotic Anal. 20 279--299.
  • Kaise, H. and Sheu, S.-J. (2004). Risk-sensitive optimal investment: Solutions of dynamical programming equation. In Mathematics of Finance, Contemporary Math. 351. Amer. Math. Soc., Providence, RI.
  • Kaise, H. and Sheu, S.-J. (2004). On the structure of solutions of ergodic type Bellman equation related to risk-sensitive control. Technical report, Academia Sinica.
  • Kaise, H. and Sheu, S.-J. (2004). Evaluation of large time expectations for diffusion processes. Technical report, Academia Sinica.
  • Ladyzhenskaya, O. A. and Ural'tseva, N. N. (1968). Linear and Quasilinear Elliptic Equations. Academic Press, New York.
  • McEneaney, W. M. and Ito, K. (1997). Infinite time-horizon risk-sensitive systems with quadratic growth. In Proceedings of 36th IEEE Conference on Decision and Control.
  • Nagai, H. (1996). Bellman equation of risk-sensitive control. SIAM J. Control Optim. 34 74--101.
  • Nagai, H. (2003). Optimal strategies for risk-sensitive portfolio optimization problems for general factor models. SIAM J. Control Optim. 41 1779--1800.
  • Pinsky, R. G. (1995). Positive Harmonic Functions and Diffusion. Cambridge Univ. Press.
  • Stroock, D. W. (1984). An Introduction to Theory of Large Deviations. Springer, Berlin.
  • Varadhan, S. R. S. (1980). Diffusion Problems and Partial Differential Equations. Springer, New York.