The Annals of Applied Probability

Discrete time approximation of fully nonlinear HJB equations via BSDEs with nonpositive jumps

Idris Kharroubi, Nicolas Langrené, and Huyên Pham

Full-text: Open access


We propose a new probabilistic numerical scheme for fully nonlinear equation of Hamilton–Jacobi–Bellman (HJB) type associated to stochastic control problem, which is based on the Feynman–Kac representation in [Kharroubi and Pham (2014)] by means of control randomization and backward stochastic differential equation with nonpositive jumps. We study a discrete time approximation for the minimal solution to this class of BSDE when the time step goes to zero, which provides both an approximation for the value function and for an optimal control in feedback form. We obtained a convergence rate without any ellipticity condition on the controlled diffusion coefficient. An explicit implementable scheme based on Monte Carlo simulations and empirical regressions, associated error analysis and numerical experiments are performed in the companion paper [ Monte Carlo Methods Appl. 20 (2014) 145–165].

Article information

Ann. Appl. Probab., Volume 25, Number 4 (2015), 2301-2338.

Received: November 2013
Revised: March 2014
First available in Project Euclid: 21 May 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 65C99: None of the above, but in this section 60J75: Jump processes 49L25: Viscosity solutions

Sample discrete time approximation Hamilton–Jacobi–Bellman equation nonlinear degenerate PDE optimal control backward stochastic differential equations


Kharroubi, Idris; Langrené, Nicolas; Pham, Huyên. Discrete time approximation of fully nonlinear HJB equations via BSDEs with nonpositive jumps. Ann. Appl. Probab. 25 (2015), no. 4, 2301--2338. doi:10.1214/14-AAP1049.

Export citation


  • [1] Bally, V. and Pagès, G. (2003). Error analysis of the optimal quantization algorithm for obstacle problems. Stochastic Process. Appl. 106 1–40.
  • [2] Barles, G., Buckdahn, R. and Pardoux, E. (1997). Backward stochastic differential equations and integral-partial differential equations. Stochastics Stochastics Rep. 60 57–83.
  • [3] Barles, G. and Jakobsen, E. R. (2007). Error bounds for monotone approximation schemes for parabolic Hamilton–Jacobi–Bellman equations. Math. Comp. 76 1861–1893 (electronic).
  • [4] Bonnans, J. F. and Zidani, H. (2003). Consistency of generalized finite difference schemes for the stochastic HJB equation. SIAM J. Numer. Anal. 41 1008–1021.
  • [5] Bouchard, B. and Chassagneux, J.-F. (2008). Discrete-time approximation for continuously and discretely reflected BSDEs. Stochastic Process. Appl. 118 2269–2293.
  • [6] Bouchard, B. and Elie, R. (2008). Discrete-time approximation of decoupled forward-backward SDE with jumps. Stochastic Process. Appl. 118 53–75.
  • [7] Bouchard, B. and Touzi, N. (2004). Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations. Stochastic Process. Appl. 111 175–206.
  • [8] Crandall, M. G., Ishii, H. and Lions, P.-L. (1992). User’s guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc. (N.S.) 27 1–67.
  • [9] Fahim, A., Touzi, N. and Warin, X. (2011). A probabilistic numerical method for fully nonlinear parabolic PDEs. Ann. Appl. Probab. 21 1322–1364.
  • [10] Fleming, W. H. and Soner, H. M. (2006). Controlled Markov Processes and Viscosity Solutions, 2nd ed. Stochastic Modelling and Applied Probability 25. Springer, New York.
  • [11] Guo, W., Zhang, J. and Zhuo, J. (2006). A monotone scheme for high dimensional fully nonlinear PDE. Preprint.
  • [12] Kharroubi, I., Langrené, N. and Pham, H. (2014). A numerical algorithm for fully nonlinear HJB equations: An approach by control randomization. Monte Carlo Methods Appl. 20 145–165.
  • [13] Kharroubi, I. and Pham, H. (2014). Feynman–Kac representation for Hamilton–Jacobi–Bellman IPDE. Ann. Probab. To appear.
  • [14] Kloeden, P. E. and Platen, E. (1992). Numerical Solution of Stochastic Differential Equations. Applications of Mathematics (New York) 23. Springer, Berlin.
  • [15] Krylov, N. V. (1999). Approximating value functions for controlled degenerate diffusion processes by using piece-wise constant policies. Electron. J. Probab. 4 1–19.
  • [16] Krylov, N. V. (2000). On the rate of convergence of finite-difference approximations for Bellman’s equations with variable coefficients. Probab. Theory Related Fields 117 1–16.
  • [17] Kushner, H. J. and Dupuis, P. G. (1992). Numerical Methods for Stochastic Control Problems in Continuous Time. Applications of Mathematics (New York) 24. Springer, New York.
  • [18] Lemor, J.-P., Gobet, E. and Warin, X. (2006). Rate of convergence of an empirical regression method for solving generalized backward stochastic differential equations. Bernoulli 12 889–916.
  • [19] Pham, H. (2009). Continuous-Time Stochastic Control and Optimization with Financial Applications. Stochastic Modelling and Applied Probability 61. Springer, Berlin.
  • [20] Possamaï, D. and Tan, X. (2013). Weak approximation of second order BSDEs. Preprint.
  • [21] Soner, H. M., Touzi, N. and Zhang, J. (2012). Wellposedness of second order backward SDEs. Probab. Theory Related Fields 153 149–190.
  • [22] Tan, X. (2013). A splitting method for fully nonlinear degenerate parabolic PDEs. Electron. J. Probab. 18 24.
  • [23] Zhang, J. (2004). A numerical scheme for BSDEs. Ann. Appl. Probab. 14 459–488.