The Annals of Applied Statistics

Parameter tuning for a multi-fidelity dynamical model of the magnetosphere

William Kleiber, Stephan R. Sain, Matthew J. Heaton, Michael Wiltberger, C. Shane Reese, and Derek Bingham

Full-text: Open access


Geomagnetic storms play a critical role in space weather physics with the potential for far reaching economic impacts including power grid outages, air traffic rerouting, satellite damage and GPS disruption. The LFM–MIX is a state-of-the-art coupled magnetospheric–ionospheric model capable of simulating geomagnetic storms. Imbedded in this model are physical equations for turning the magnetohydrodynamic state parameters into energy and flux of electrons entering the ionosphere, involving a set of input parameters. The exact values of these input parameters in the model are unknown, and we seek to quantify the uncertainty about these parameters when model output is compared to observations. The model is available at different fidelities: a lower fidelity which is faster to run, and a higher fidelity but more computationally intense version. Model output and observational data are large spatiotemporal systems; the traditional design and analysis of computer experiments is unable to cope with such large data sets that involve multiple fidelities of model output. We develop an approach to this inverse problem for large spatiotemporal data sets that incorporates two different versions of the physical model. After an initial design, we propose a sequential design based on expected improvement. For the LFM–MIX, the additional run suggested by expected improvement diminishes posterior uncertainty by ruling out a posterior mode and shrinking the width of the posterior distribution. We also illustrate our approach using the Lorenz ‘96 system of equations for a simplified atmosphere, using known input parameters. For the Lorenz ‘96 system, after performing sequential runs based on expected improvement, the posterior mode converges to the true value and the posterior variability is reduced.

Article information

Ann. Appl. Stat., Volume 7, Number 3 (2013), 1286-1310.

First available in Project Euclid: 3 October 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Computer experiments expected improvement geomagnetic storm inverse problem Lorenz ‘96 model fidelity sequential design uncertainty quantification


Kleiber, William; Sain, Stephan R.; Heaton, Matthew J.; Wiltberger, Michael; Reese, C. Shane; Bingham, Derek. Parameter tuning for a multi-fidelity dynamical model of the magnetosphere. Ann. Appl. Stat. 7 (2013), no. 3, 1286--1310. doi:10.1214/13-AOAS651.

Export citation


  • Banerjee, S., Gelfand, A. E., Finley, A. O. and Sang, H. (2008). Gaussian predictive process models for large spatial data sets. J. R. Stat. Soc. Ser. B Stat. Methodol. 70 825–848.
  • Bayarri, M. J., Berger, J. O., Paulo, R., Sacks, J., Cafeo, J. A., Cavendish, J., Lin, C.-H. and Tu, J. (2007). A framework for validation of computer models. Technometrics 49 138–154.
  • Bhat, K. S., Haran, M. and Goes, M. (2010). Computer model calibration with multivariate spatial output: A case study in climate parameter learning. In Frontiers of Statistical Decision Making and Bayesian Analysis (M. H. Chen, P. Müller, D. Sun, K. Ye and D. K. Dey, eds.) 401–408. Springer, New York.
  • De Cesare, L., Myers, D. E. and Posa, D. (2001). Estimating and modeling space–time correlation structures. Statist. Probab. Lett. 51 9–14.
  • De Iaco, S., Myers, D. E. and Posa, D. (2001). Space–time analysis using a general product-sum model. Statist. Probab. Lett. 52 21–28.
  • Forrester, A. I. J., Sóbester, A. and Keane, A. J. (2007). Multi-fidelity optimization via surrogate modelling. Proc. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 463 3251–3269.
  • Fuentes, M. (2006). Testing for separability of spatial–temporal covariance functions. J. Statist. Plann. Inference 136 447–466.
  • Guttorp, P. and Gneiting, T. (2006). Studies in the history of probability and statistics. XLIX. On the Matérn correlation family. Biometrika 93 989–995.
  • Heaton, M. J., Kleiber, W., Sain, S. R. and Wiltberger, M. (2013). Emulating and calibrating the multiple-fidelity Lyon–Fedder–Mobarry magnetosphere–ionosphere coupled computer model. Unpublished manuscript.
  • Higdon, D., Kennedy, M., Cavendish, J. C., Cafeo, J. A. and Ryne, R. D. (2004). Combining field data and computer simulations for calibration and prediction. SIAM J. Sci. Comput. 26 448–466.
  • Higdon, D., Gattiker, J., Williams, B. and Rightley, M. (2008a). Computer model calibration using high-dimensional output. J. Amer. Statist. Assoc. 103 570–583.
  • Higdon, D., Nakhleh, C., Gattiker, J. and Williams, B. (2008b). A Bayesian calibration approach to the thermal problem. Comput. Methods Appl. Mech. Engrg. 197 2431–2441.
  • Higdon, D., Heitmann, K., Lawrence, E. and Habib, S. (2011). Using the Bayesian framework to combine simulations and physical observations. In Large-Scale Inverse Problems and Quantification of Uncertainty (L. Biegler, G. Biros, O. Ghattas, M. Heinkenschloss, D. Keyes, B. Mallick, L. Tenorio, B. van Bloemen Waanders, K. Willcox and Y. Marzouk, eds.) 87–106. Wiley, Chichester.
  • Johnson, M. E., Moore, L. M. and Ylvisaker, D. (1990). Minimax and maximin distance designs. J. Statist. Plann. Inference 26 131–148.
  • Jones, D. R., Schonlau, M. and Welch, W. J. (1998). Efficient global optimization of expensive black-box functions. J. Global Optim. 13 455–492.
  • Kennedy, M. C. and O’Hagan, A. (2000). Predicting the output from a complex computer code when fast approximations are available. Biometrika 87 1–13.
  • Kennedy, M. C. and O’Hagan, A. (2001). Bayesian calibration of computer models. J. R. Stat. Soc. Ser. B Stat. Methodol. 63 425–464.
  • Le Gratiet, L. (2012). Bayesian analysis of hierarchical multi-fidelity codes. Available at arXiv:1112.5389v2 [math.ST].
  • Lorenz, E. N. (1996). Predictability—A problem partly solved, 1–18. Reading, United Kingdom, ECMWF.
  • Lorenz, E. N. (2005). Designing chaotic models. J. Atmospheric Sci. 62 1574–1587.
  • Lyon, J. G., Fedder, J. A. and Mobarry, C. M. (2004). The Lyon–Fedder–Mobarry (LFM) global MHD magnetospheric simulation code. Journal of Atmospheric and Solar–Terrestrial Physics 66 1333–1350.
  • Mitchell, M. W., Genton, M. G. and Gumpertz, M. L. (2005). Testing for separability of space–time covariances. Environmetrics 16 819–831.
  • National Research Council (2008). Severe space weather events—Understanding societal and economic impacts: A workshop report. National Academies Press, Washington, DC.
  • Pratola, M. T., Sain, S. R., Bingham, D., Wiltberger, M. and Rigler, J. (2013). Fast sequential computer model calibration of large non-stationary spatial–temporal processes. Technometrics 55 232–242.
  • Qian, P. Z. G. and Wu, C. F. J. (2008). Bayesian hierarchical modeling for integrating low-accuracy and high-accuracy experiments. Technometrics 50 192–204.
  • Qian, Z., Seepersad, C. C., Joseph, V. R., Allen, J. K. and Wu, C. F. J. (2006). Building surrogate models based on detailed and approximate simulations. Journal of Mechanical Design 128 668–677.
  • Rougier, J. (2008). Efficient emulators for multivariate deterministic functions. J. Comput. Graph. Statist. 17 827–843.
  • Rougier, J., Guillas, S., Maute, A. and Richmond, A. D. (2009). Expert knowledge and multivariate emulation: The thermosphere–ionosphere electrodynamics general circulation model (TIE–GCM). Technometrics 51 414–424.
  • Sacks, J., Welch, W. J., Mitchell, T. J. and Wynn, H. P. (1989). Design and analysis of computer experiments. Statist. Sci. 4 409–435.
  • Santner, T. J., Williams, B. J. and Notz, W. I. (2003). The Design and Analysis of Computer Experiments. Springer, New York.
  • Tarantola, A. (2005). Inverse Problem Theory and Methods for Model Parameter Estimation. SIAM, Philadelphia, PA.
  • Wikle, C. K. (2010). Low-rank representations for spatial processes. In Handbook of Spatial Statistics 107–118. CRC Press, Boca Raton, FL.
  • Wilkinson, R. D. (2010). Bayesian calibration of expensive multivariate computer experiments. In Large-Scale Inverse Problems and Quantification of Uncertainty (L. Biegler, G. Biros, O. Ghattas, M. Heinkenschloss, D. Keyes, B. Mallick, L. Tenorio, B. van Bloemen Waanders, K. Willcox and Y. Marzouk, eds.). Wiley, New York.
  • Wiltberger, M., Wang, W., Burns, A. G., Solomon, S. C., Lyon, J. G. and Goodrich, C. C. (2004). Initial results from the coupled magnetosphere ionosphere thermosphere model: Magnetospheric and ionospheric responses. Journal of Atmospheric and Solar-Terrestrial Physics 66 1411–1423.
  • Wiltberger, M., Weigel, R. S., Lotko, W. and Fedder, J. A. (2009). Modeling seasonal variations of auroral particle precipitation in a global-scale magnetosphere–ionosphere simulation. Journal of Geophysical Research 114 A01204.