• Bernoulli
  • Volume 23, Number 4B (2017), 3537-3570.

Efficient Bayesian estimation and uncertainty quantification in ordinary differential equation models

Prithwish Bhaumik and Subhashis Ghosal

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


Often the regression function is specified by a system of ordinary differential equations (ODEs) involving some unknown parameters. Typically analytical solution of the ODEs is not available, and hence likelihood evaluation at many parameter values by numerical solution of equations may be computationally prohibitive. Bhaumik and Ghosal (Electron. J. Stat. 9 (2015) 3124–3154) considered a Bayesian two-step approach by embedding the model in a larger nonparametric regression model, where a prior is put through a random series based on B-spline basis functions. A posterior on the parameter is induced from the regression function by minimizing an integrated weighted squared distance between the derivative of the regression function and the derivative suggested by the ODEs. Although this approach is computationally fast, the Bayes estimator is not asymptotically efficient. In this paper, we suggest a modification of the two-step method by directly considering the distance between the function in the nonparametric model and that obtained from a four stage Runge–Kutta (RK$4$) method. We also study the asymptotic behavior of the posterior distribution of $\mathbf{\theta}$ based on an approximate likelihood obtained from an RK$4$ numerical solution of the ODEs. We establish a Bernstein–von Mises theorem for both methods which assures that Bayesian uncertainty quantification matches with the frequentist one and the Bayes estimator is asymptotically efficient.

Article information

Bernoulli, Volume 23, Number 4B (2017), 3537-3570.

Received: November 2014
Revised: April 2016
First available in Project Euclid: 23 May 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

approximate likelihood Bayesian inference Bernstein–von Mises theorem ordinary differential equation Runge–Kutta method spline smoothing


Bhaumik, Prithwish; Ghosal, Subhashis. Efficient Bayesian estimation and uncertainty quantification in ordinary differential equation models. Bernoulli 23 (2017), no. 4B, 3537--3570. doi:10.3150/16-BEJ856.

Export citation


  • [1] Anderson, R.M. and May, R.M. (1992). Infectious Diseases of Humans: Dynamics and Control. London: Oxford Univ. Press.
  • [2] Bhaumik, P. and Ghosal, S. (2015). Bayesian two-step estimation in differential equation models. Electron. J. Stat. 9 3124–3154.
  • [3] Brunel, N.J.-B. (2008). Parameter estimation of ODE’s via nonparametric estimators. Electron. J. Stat. 2 1242–1267.
  • [4] Brunel, N.J.-B., Clairon, Q. and d’Alché-Buc, F. (2014). Parametric estimation of ordinary differential equations with orthogonality conditions. J. Amer. Statist. Assoc. 109 173–185.
  • [5] Campbell, D. and Steele, R.J. (2012). Smooth functional tempering for nonlinear differential equation models. Stat. Comput. 22 429–443.
  • [6] Chen, T., He, H.L. and Church, G.M. (1999). Modeling gene expression with differential equations. In Pacific Symposium on Biocomputing 4 4.
  • [7] de Boor, C. (1978). A Practical Guide to Splines. Applied Mathematical Sciences 27. New York: Springer.
  • [8] Gabrielsson, J. and Weiner, D. (2000). Pharmacokinetic and Pharmacodynamic Data Analysis: Concepts and Applications. London: Taylor & Francis.
  • [9] Gelman, A., Bois, F. and Jiang, J. (1996). Physiological pharmacokinetic analysis using population modeling and informative prior distributions. J. Amer. Statist. Assoc. 91 1400–1412.
  • [10] Ghosh, S.K. and Goyal, L. (2010). Statistical inference for non-linear models involving ordinary differential equations. J. Stat. Theory Pract. 4 727–742.
  • [11] Girolami, M. (2008). Bayesian inference for differential equations. Theoret. Comput. Sci. 408 4–16.
  • [12] Gugushvili, S. and Klaassen, C.A.J. (2012). $\sqrt{n}$-consistent parameter estimation for systems of ordinary differential equations: Bypassing numerical integration via smoothing. Bernoulli 18 1061–1098.
  • [13] Hairer, E., Nørsett, S.P. and Wanner, G. (1993). Solving Ordinary Differential Equations. I: Nonstiff Problems, 2nd ed. Springer Series in Computational Mathematics 8. Berlin: Springer.
  • [14] Henrici, P. (1962). Discrete Variable Methods in Ordinary Differential Equations. New York: Wiley.
  • [15] Jaeger, J. (2012). Functional estimation in systems defined by differential equation using Bayesian smoothing methods. Ph.D. thesis, UCL.
  • [16] Kleijn, B.J.K. and van der Vaart, A.W. (2006). Misspecification in infinite-dimensional Bayesian statistics. Ann. Statist. 34 837–877.
  • [17] Kleijn, B.J.K. and van der Vaart, A.W. (2012). The Bernstein–Von-Mises theorem under misspecification. Electron. J. Stat. 6 354–381.
  • [18] Mattheij, R. and Molenaar, J. (2002). Ordinary Differential Equations in Theory and Practice. Classics in Applied Mathematics 43. Philadelphia, PA: SIAM.
  • [19] Nowak, M.A. and May, R.M. (2000). Virus Dynamics: Mathematical Principles of Immunology and Virology. Oxford: Oxford Univ. Press.
  • [20] Qi, X. and Zhao, H. (2010). Asymptotic efficiency and finite-sample properties of the generalized profiling estimation of parameters in ordinary differential equations. Ann. Statist. 38 435–481.
  • [21] Ramsay, J.O., Hooker, G., Campbell, D. and Cao, J. (2007). Parameter estimation for differential equations: A generalized smoothing approach. J. R. Stat. Soc. Ser. B Stat. Methodol. 69 741–796.
  • [22] Rodriguez-Fernandez, M., Egea, J.A. and Banga, J.R. (2006). Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems. BMC Bioinformatics 7 483.
  • [23] Rogers, S., Khanin, R. and Girolami, M. (2007). Bayesian model-based inference of transcription factor activity. BMC Bioinformatics 8 S2.
  • [24] Storn, R. and Price, K. (1997). Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11 341–359.
  • [25] van der Vaart, A.W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge: Cambridge Univ. Press.
  • [26] Varah, J.M. (1982). A spline least squares method for numerical parameter estimation in differential equations. SIAM J. Sci. Statist. Comput. 3 28–46.
  • [27] Wu, H., Xue, H. and Kumar, A. (2012). Numerical discretization-based estimation methods for ordinary differential equation models via penalized spline smoothing with applications in biomedical research. Biometrics 68 344–352.
  • [28] Xue, H., Miao, H. and Wu, H. (2010). Sieve estimation of constant and time-varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error. Ann. Statist. 38 2351–2387.
  • [29] Zhou, S., Shen, X. and Wolfe, D.A. (1998). Local asymptotics for regression splines and confidence regions. Ann. Statist. 26 1760–1782.
  • [30] Zhou, S. and Wolfe, D.A. (2000). On derivative estimation in spline regression. Statist. Sinica 10 93–108.