Electronic Journal of Statistics

Empirical evolution equations

Susan Wei and Victor M. Panaretos

Full-text: Open access


Evolution equations comprise a broad framework for describing the dynamics of a system in a general state space: when the state space is finite-dimensional, they give rise to systems of ordinary differential equations; for infinite-dimensional state spaces, they give rise to partial differential equations. Several modern statistical and machine learning methods concern the estimation of objects that can be formalized as solutions to evolution equations, in some appropriate state space, even if not stated as such. The corresponding equations, however, are seldom known exactly, and are empirically derived from data, often by means of non-parametric estimation. This induces uncertainties on the equations and their solutions that are challenging to quantify, and moreover the diversity and the specifics of each particular setting may obscure the path for a general approach. In this paper, we address the problem of constructing general yet tractable methods for quantifying such uncertainties, by means of asymptotic theory combined with bootstrap methodology. We demonstrates these procedures in important examples including gradient line estimation, diffusion tensor imaging tractography, and local principal component analysis. The bootstrap perspective is particularly appealing as it circumvents the need to simulate from stochastic (partial) differential equations that depend on (infinite-dimensional) unknowns. We assess the performance of the bootstrap procedure via simulations and find that it demonstrates good finite-sample coverage.

Article information

Electron. J. Statist., Volume 12, Number 1 (2018), 249-276.

Received: August 2017
First available in Project Euclid: 12 February 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Diffusion tensor imaging gradient line heat flow integral curve local principal curve scale space vector field

Creative Commons Attribution 4.0 International License.


Wei, Susan; Panaretos, Victor M. Empirical evolution equations. Electron. J. Statist. 12 (2018), no. 1, 249--276. doi:10.1214/17-EJS1382. https://projecteuclid.org/euclid.ejs/1518426110

Export citation


  • Arias-Castro, E., Mason, D. and Pelletier, B. (2016). On the Estimation of the Gradient Lines of a Density and the Consistency of the Mean-Shift Algorithm., Journal of Machine Learning Research 17 1–28.
  • Atkinson, K. and Han, W. (2009). Linear Operators on Normed Spaces. In, Theoreticl Numerical Analysis: A Functional Analysis Framework 51–113.
  • Basser, P. J., Mattiello, J. and LeBihan, D. (1994). MR diffusion tensor spectroscopy and imaging., Biophysical journal 66 259–67.
  • Bellman, R. (2008)., Stability Theory of Differential Equations. Dover Publications, New York.
  • Bierens, H. J. (1987). Kernel estimators of regression functions. In, Advances in econometrics: Fifth world congress 99–144.
  • Blanes, S., Casas, F., Oteo, J. a. and Ros, J. (2009). The Magnus expansion and some of its applications., Physics Reports 470 151–238.
  • Bottou, L. (2010)., Large-Scale Machine Learning with Stochastic Gradient Descent In Proceedings of COMPSTAT’2010: 19th International Conference on Computational Statistics Paris France, August 22-27, 2010 Keynote, Invited and Contributed Papers 177–186. Physica-Verlag HD, Heidelberg.
  • Brunel, N. J.-B., Clairon, Q. and D’Alché-Buc, F. (2014). Parametric Estimation of Ordinary Differential Equations With Orthogonality Conditions., Journal of the American Statistical Association 109 173–185.
  • Chandra, J. (1970). On a Generalization of the Gronwall-Bellman in Partially Ordered Banach Spaces., Journal of Mathematical Analysis and Applications 681 668–681.
  • Chaudhuri, P. and Marron, J. S. (2000). Scale Space View of Curve Estimation., The Annals of Statistics 28 408–428.
  • Chen, Y.-C. C., Genovese, C. R. and Wasserman, L. (2015). Asymptotic theory for density ridges., Annals of Statistics 43 1896–1928.
  • Chen, Y.-C., Genovese, C. R. and Wasserman, L. (2016). A Comprehensive Approach to Mode Clustering., Electronic Journal of Statistics 10 210–241.
  • Chen, Y. C., Genovese, C. R. and Wasserman, L. (2017a). Density Level Sets: Asymptotics, Inference, and Visualization., Journal of the American Statistical Association 1–13.
  • Chen, Y. C., Genovese, C. R. and Wasserman, L. (2017b). Statistical inference using the morse-smale complex., Electronic Journal of Statistics 11 1390–1433.
  • Chen, Y.-C., Genovese, C. R., Ho, S. and Wasserman, L. (2015). Optimal Ridge Detection using Coverage Risk. In, Advances in Neural Information Processing Systems 28 316–324.
  • Ciollaro, M., Genovese, C. R. and Wang, D. (2016). Nonparametric Clustering of Functional Data Using, Pseudo-Densities.
  • Delaigle, A. and Hall, P. (2010). Defining probability density for a distribution of random functions., Annals of Statistics 38 1171–1193.
  • Efron, B. (1979). Bootstrap Methods: Another Look at the Jackknife., The Annals of Statistics 7 1–26.
  • Efron, B. (1982)., The jackknife, the bootstrap, and other resampling plans. Society for Industrial and Applied Mathematics.
  • Einbeck, J. and Dwyer, J. (2011). Using principal curves to analyse traffic patterns on freeways., Transportmetrica 7 229–246.
  • Einbeck, J., Evers, L. and Bailer-Jones, C. (2008). Representing complex data using localized principal components with application to astronomical data. In, Lecture Notes in Computational Science and Engineering 58 178–201.
  • Einbeck, J., Tutz, G. and Evers, L. (2005). Local principal curves., Statistics and Computing 15 301–313.
  • Genovese, C. R., Perone-Pacifico, M., Verdinelli, I. and Wasserman, L. (2009). On the path density of a gradient field., Annals of Statistics 37 3236–3271.
  • Hageman, N. S., Toga, A. W., Narr, K. L. and Shattuck, D. W. (2009). A diffusion tensor imaging tractography algorithm based on Navier-Stokes fluid mechanics., IEEE Transactions on Medical Imaging 28 348–60.
  • Hall, P. and Heckman, N. E. (2002). Estimating and depicting the structure of a distribution of random functions., Biometrika 89 145–158.
  • Hastie, T. and Stuetzle, W. (1989). Principal Curves., Journal of the American Statistical Association 84 502–516.
  • Hill, B. J., Kendall, W. S. and Thönnes, E. (2012). Fibre-generated point processes and fields of orientations., Annals of Applied Statistics 6 994–1020.
  • Huckemann, S., Hotz, T. and Munk, A. (2008). Global models for the orientation field of fingerprints: An approach based on quadratic differentials., IEEE Transactions on Pattern Analysis and Machine Intelligence 30 1507–1519.
  • Koltchinskii, V., Sakhanenko, L. and Cai, S. (2007). Integral curves of noisy vector fields and statistical problems in diffusion tensor imaging: nonparametric kernel estimation and hypotheses testing., Annals of Statistics 35 1576–1607.
  • Kosorok, M. R. (2008). Introduction to Empirical Processes and Semiparametric Inference., Springer August 1–491.
  • Krein, S. G. (1971)., Linear differential equations in Banach space. American Mathematical Society, Providence, R.I.
  • Lee, K. K. (1992)., Lectures on Dynamical Systems, Structural Stability and Their Applications. World Scientific, Singapore.
  • Lee, J. M. (2012)., Introduction to Smooth Manifolds, 2 ed. Springer New York, New York.
  • Lisini, S. (2009). Nonlinear diffusion equations with variable coefficients as gradient flows in Wasserstein spaces., ESAIM: Control, Optimisation and Calculus of Variations 15 712–740.
  • Mikusiński, J. (1978)., The Bochner integral. Birkhauser Verlag.
  • Moler, C. andVan Loan, C. (2003). Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later., SIAM Review 45 3–49.
  • Mukherjee, P., Berman, J. I., Chung, S. W., Hess, C. P. and Henry, R. G. (2008). Diffusion tensor MR imaging and fiber tractography: theoretic underpinnings., American Journal of Neuroradiology 29 632–41.
  • Nelson, E. (1969)., Topics in Dynamics I: Flows. Princeton University Press, Princeton.
  • Ozertem, U. and Erdogmus, D. (2011). Locally Defined Principal Curves and Surfaces., Journal of Machine Learning Research 12 1249–1286.
  • Panaretos, V. M., Pham, T. and Yao, Z. (2014). Principal flows., Journal of the American Statistical Association 109 424–436.
  • Praestgaard, J. and Wellner, J. A. (1993). Exchangeably Weighted Bootstraps of the General Empirical Process., The Annals of Probability 21 2053–2086.
  • Ramsay, J. O., Hooker, G., Campbell, D. and Cao, J. (2007). Parameter Estimation for Differential Equations: A Generalized Smoothing Approach., Differential Equations 69 741–796.
  • Rinaldo, A. and Nugent, R. (2012). Stability of Density-Based Clustering., Journal of Machine Learning Reasearch 13 905–948.
  • Rinaldo, A. and Wasserman, L. (2010). Generalized density clustering., Annals of Statistics 38 2678–2722.
  • Theisel, H., Weinkauf, T., Hege, H. C. and Seidel, H. P. (2004). Stream line and path line oriented topology for 2D time-dependent vector fields., IEEE Visualization 2004 - Proceedings, VIS 2004 321–328.
  • van der Vaart, A. W. and Wellner, J. A. (1996)., Weak Convergence and Empirical Processes. Springer series in statistics. Springer.
  • Vanhems, A. (2006). Nonparametric study of solutions of differential equations., Econometric Theory 22 127–157.
  • Walker, J. A. (1980)., Dynamical Systems and Evolution Equations. Springer, Boston.
  • Zachmanoglou, E. C. and Thoe, D. W. (1986)., Introduction to partial differential equations with applications. Dover Publications, New York.