Electronic Journal of Probability

Degenerate Variance Control in the One-dimensional Stationary Case

Daniel Ocone and Ananda Weerasinghe

Full-text: Open access

Abstract

We study the problem of stationary control by adaptive choice of the diffusion coefficient in the case that control degeneracy is allowed and the drift admits a unique, asymptotically stable equilibrium point. We characterize the optimal value and obtain it as an Abelian limit of optimal discounted values and as a limiting average of finite horizon optimal values, and we also characterize the optimal stationary strategy. In the case of linear drift, the optimal stationary value is expressed in terms of the solution of an optimal stopping problem. We generalize the above results to allow unbounded cost functions.

Article information

Source
Electron. J. Probab., Volume 8 (2003), paper no. 24, 27 p.

Dates
First available in Project Euclid: 23 May 2016

Permanent link to this document
https://projecteuclid.org/euclid.ejp/1464037597

Digital Object Identifier
doi:10.1214/EJP.v8-181

Mathematical Reviews number (MathSciNet)
MR2041825

Zentralblatt MATH identifier
1129.93548

Subjects
Primary: 93E20: Optimal stochastic control
Secondary: 60G35: Signal detection and filtering [See also 62M20, 93E10, 93E11, 94Axx]

Keywords
stochastic control stationary control degenerate variance control

Citation

Ocone, Daniel; Weerasinghe, Ananda. Degenerate Variance Control in the One-dimensional Stationary Case. Electron. J. Probab. 8 (2003), paper no. 24, 27 p. doi:10.1214/EJP.v8-181. https://projecteuclid.org/euclid.ejp/1464037597


Export citation

References

  • Assaf, D. (1997), Estimating the state of a noisy continuous time Markov chain when dynamic sampling is feasible, Ann. Appl. Probab. 3, 822-836.
  • Arisawa, M. and Lions, P-L. (1998), On ergodic stochastic control, Comm. Partial Differential Equations 23, 2187-2217.
  • Basak, G.K. and Borkar, V.S. and Ghosh, M.K. (1997), Ergodic control of degenerate diffusions, Stochastic Anal. Appl. 15, 1-17.
  • Benes, V.E. and Karatzas, I. (1981), Optimal stationary linear control of the Wiener process, J. of Optimization Theory and Applications 35, 611-633.
  • Bensoussan, A. and Frehse, J. (1992), On Bellman equations of ergodic control in Rn J. Reine Angew. Math. 429, 125-160.
  • Bensoussan, A. and Lions, J.L. (1982) Applications of Variational Inequalities in Stochastic Control, North Holland, Amsterdam and New York.
  • Borkar, V.S. (1995), On ergodic control of degenerate diffusions, J. Optim. Theory and Appl. 86, 251-265.
  • Borkar, V.S. and Ghosh, M.K. (1998) Ergodic control of multi-dimensional diffusions I. The existence results, SIAM J.Control Optimization 26, 112-126.
  • Cox, R.M. (1984) Stationary and Discounted Stochastic Control, Ph.D. thesis, Statistics Dept., Columbia University.
  • Cox, R.M. and Karatzas, I. (1985) Stationary control of Brownian motion in several dimensions, Adv. Appl. Prob. 17, 531-561.
  • Krylov, N.V. (1973), Control of a solution of a stochastic integral equation with degeneration, Theory of Probability and Its Applications 17, 114-131.
  • Kurtz, T.G. and Stockbridge, R.H. (1998), Existence of Markov controls and characterization of optimal controls, SIAM J.Control Optimization 36, 609-653.
  • Kushner, H.J. (1978), Optimality conditions for the average cost per unit time problem with a diffusion model, SIAM J.Control Optimization 16, 330-346.
  • Kushner, H.J. (1999), Existence of optimal controls for variance control, in Stochastic Analysis, Control, Optimization, and Applications: a volume in honor of W.H. Fleming, edited by McEneany, W.H. and Yin, G.G. and Zhang, Q., Birkhäuser, Boston, 421-437.
  • Lions, P-L.(1981) Control of diffusions in $R^n$, Comm. Pure Appl. Math. 34, 121-147.
  • Mandl, P. (1968), Analytical Treatment of One-Dimensional Markov Processes, Springer-Verlag, New York.
  • Ocone, D. and Weerasinghe, A. (2000), Degenerate variance control of a one-dimensional diffusion, SIAM J.Control Optimization 39, 1-24.
  • Tarres, R. (1988), Asymptotic evolution of a stochastic control problem, SIAM J.Control Optimization 23, 614-631.