The Annals of Applied Probability

Optimal control of branching diffusion processes: A finite horizon problem

Julien Claisse

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

In this paper, we aim to develop the stochastic control theory of branching diffusion processes where both the movement and the reproduction of the particles depend on the control. More precisely, we study the problem of minimizing the expected value of the product of individual costs penalizing the final position of each particle. In this setting, we show that the value function is the unique viscosity solution of a nonlinear parabolic PDE, that is, the Hamilton–Jacobi–Bellman equation corresponding to the problem. To this end, we extend the dynamic programming approach initiated by Nisio [J. Math. Kyoto Univ. 25 (1985) 549–575] to deal with the lack of independence between the particles as well as between the reproduction and the movement of each particle. In particular, we exploit the particular form of the optimization criterion to derive a weak form of the branching property. In addition, we provide a precise formulation and a detailed justification of the adequate dynamic programming principle.

Article information

Source
Ann. Appl. Probab., Volume 28, Number 1 (2018), 1-34.

Dates
Received: September 2016
Revised: December 2016
First available in Project Euclid: 3 March 2018

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1520046082

Digital Object Identifier
doi:10.1214/17-AAP1290

Mathematical Reviews number (MathSciNet)
MR3770871

Zentralblatt MATH identifier
06873678

Subjects
Primary: 93E20: Optimal stochastic control 60J60: Diffusion processes [See also 58J65] 60J80: Branching processes (Galton-Watson, birth-and-death, etc.)
Secondary: 49L20: Dynamic programming method 49L25: Viscosity solutions 60J70: Applications of Brownian motions and diffusion theory (population genetics, absorption problems, etc.) [See also 92Dxx] 60J85: Applications of branching processes [See also 92Dxx]

Keywords
Stochastic control branching diffusion process dynamic programming principle Hamilton–Jacobi–Bellman equation viscosity solution

Citation

Claisse, Julien. Optimal control of branching diffusion processes: A finite horizon problem. Ann. Appl. Probab. 28 (2018), no. 1, 1--34. doi:10.1214/17-AAP1290. https://projecteuclid.org/euclid.aoap/1520046082


Export citation

References

  • [1] Bansaye, V. and Tran, V. C. (2011). Branching Feller diffusion for cell division with parasite infection. ALEA Lat. Am. J. Probab. Math. Stat. 8 95–127.
  • [2] Barles, G. and Souganidis, P. E. (1991). Convergence of approximation schemes for fully nonlinear second order equations. Asymptot. Anal. 4 271–283.
  • [3] Bellman, R. (1957). Dynamic Programming. Princeton Univ. Press, Princeton, NJ.
  • [4] Bismut, J.-M. (1976). Théorie probabiliste du contrôle des diffusions. Mem. Amer. Math. Soc. 4 xiii+130.
  • [5] Borkar, V. S. (1989). Optimal Control of Diffusion Processes. Pitman Research Notes in Mathematics Series 203. Longman Scientific & Technical, Harlow.
  • [6] Ceci, C. and Gerardi, A. (1999). Optimal control and filtering of the reproduction law of a branching process. Acta Appl. Math. 55 27–50.
  • [7] Champagnat, N. and Claisse, J. (2016). On the link between infinite horizon control and quasi-stationary distributions. Preprint. Available at arXiv:1607.08046.
  • [8] Champagnat, N. and Méléard, S. (2007). Invasion and adaptive evolution for individual-based spatially structured populations. J. Math. Biol. 55 147–188.
  • [9] Crandall, M. G., Ishii, H. and Lions, P.-L. (1992). User’s guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc. (N.S.) 27 1–67.
  • [10] Davis, M. H. A. (1993). Markov Models and Optimization. Monographs on Statistics and Applied Probability 49. Chapman & Hall, London.
  • [11] Dawson, D. A. (1993). Measure-valued Markov processes. In École d’Été de Probabilités de Saint-Flour XXI—1991. Lecture Notes in Math. 1541 1–260. Springer, Berlin.
  • [12] El Karoui, N., Hu̇u̇ Nguyen, D. and Jeanblanc-Picqué, M. (1987). Compactification methods in the control of degenerate diffusions: Existence of an optimal control. Stochastics 20 169–219.
  • [13] Fleming, W. H. and Soner, H. M. (2006). Controlled Markov Processes and Viscosity Solutions, 2nd ed. Stochastic Modelling and Applied Probability 25. Springer, New York.
  • [14] Guo, X. and Hernández-Lerma, O. (2009). Continuous-Time Markov Decision Processes: Theory and Applications. Stochastic Modelling and Applied Probability 62. Springer, Berlin.
  • [15] Henry-Labordère, P., Tan, X. and Touzi, N. (2014). A numerical algorithm for a class of BSDEs via the branching process. Stochastic Process. Appl. 124 1112–1140.
  • [16] Ikeda, N., Nagasawa, M. and Watanabe, S. (1968). Branching Markov processes. I. J. Math. Kyoto Univ. 8 233–278.
  • [17] Ikeda, N., Nagasawa, M. and Watanabe, S. (1968). Branching Markov processes. II. J. Math. Kyoto Univ. 8 365–410.
  • [18] Ikeda, N., Nagasawa, M. and Watanabe, S. (1969). Branching Markov processes. III. J. Math. Kyoto Univ. 9 95–160.
  • [19] Ikeda, N. and Watanabe, S. (1989). Stochastic Differential Equations and Diffusion Processes, 2nd ed. North-Holland Mathematical Library 24. North-Holland, Amsterdam.
  • [20] Karatzas, I. and Shreve, S. E. (1991). Brownian Motion and Stochastic Calculus, 2nd ed. Graduate Texts in Mathematics 113. Springer, New York.
  • [21] Krylov, N. V. (1980). Controlled Diffusion Processes. Applications of Mathematics 14. Springer, New York.
  • [22] Krylov, N. V. (1987). Nonlinear Elliptic and Parabolic Equations of the Second Order. Mathematics and Its Applications (Soviet Series) 7. Reidel, Dordrecht.
  • [23] Lenhart, S. and Workman, J. T. (2007). Optimal Control Applied to Biological Models. Chapman & Hall/CRC, Boca Raton, FL.
  • [24] Lions, P. L. (1983). Optimal control of diffusion processes and Hamilton–Jacobi–Bellman equations. II. Viscosity solutions and uniqueness. Comm. Partial Differential Equations 8 1229–1276.
  • [25] McNamara, J. M., Houston, A. I. and Collins, E. J. (2001). Optimality models in behavioral biology. SIAM Rev. 43 413–466 (electronic).
  • [26] Nisio, M. (1985). Stochastic control related to branching diffusion processes. J. Math. Kyoto Univ. 25 549–575.
  • [27] Øksendal, B. and Sulem, A. (2005). Applied Stochastic Control of Jump Diffusions. Springer, Berlin.
  • [28] Pham, H. (2009). Continuous-Time Stochastic Control and Optimization with Financial Applications. Stochastic Modelling and Applied Probability 61. Springer, Berlin.
  • [29] Puterman, M. L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, Wiley, New York.
  • [30] Roelly, S. and Rouault, A. (1990). Construction et propriétés de martingales des branchements spatiaux interactifs. Int. Stat. Rev. 58 173–189.
  • [31] Roelly-Coppoletta, S. (1986). A criterion of convergence of measure-valued processes: Application to measure branching processes. Stochastics 17 43–65.
  • [32] Sawyer, S. (1976). Branching diffusion processes in population genetics. Adv. in Appl. Probab. 8 659–689.
  • [33] Skorohod, A. V. (1964). Branching diffusion processes. Teor. Verojatnost. i Primenen. 9 492–497.
  • [34] Touzi, N. (2013). Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE. Fields Institute Monographs 29. Springer, New York.
  • [35] Ustunel, S. (1981). Construction of branching diffusion processes and their optimal stochastic control. Appl. Math. Optim. 7 11–33.
  • [36] Zhan, Y. (1999). Viscosity solutions of nonlinear degenerate parabolic equations and several applications. Ph.D. thesis, Univ. Toronto.