Electronic Journal of Statistics

Bayes linear analysis of risks in sequential optimal design problems

Matthew Jones, Michael Goldstein, Philip Jonathan, and David Randell

Full-text: Open access


In a statistical or physical model, it is often the case that a set of design inputs must be selected in order to perform an experiment to collect data with which to update beliefs about a set of model parameters; frequently, the model also depends on a set of external variables which are unknown before the experiment is carried out, but which cannot be controlled. Sequential optimal design problems are concerned with selecting these design inputs in stages (at different points in time), such that the chosen design is optimal with respect to the set of possible outcomes of all future experiments which might be carried out. Such problems are computationally expensive.

We consider the calculations which must be performed in order to solve a sequential design problem, and we propose a framework using Bayes linear emulators to approximate all difficult calculations which arise; these emulators are designed so that we can easily approximate expectations of the risk by integrating the emulator directly, and so that we can efficiently search the design input space for settings which may be optimal. We also consider how the structure of the design calculation can be exploited to improve the quality of the fitted emulators. Our framework is demonstrated through application to a simple linear modelling problem, and to a more complex airborne sensing problem, in which a sequence of aircraft flight paths must be designed so as to collect data which are informative for the locations of ground-based gas sources.

Article information

Electron. J. Statist., Volume 12, Number 2 (2018), 4002-4031.

Received: January 2018
First available in Project Euclid: 11 December 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Bayesian uncertainty analysis emulation sequential experimental design decision support remote sensing uncertainty quantification

Creative Commons Attribution 4.0 International License.


Jones, Matthew; Goldstein, Michael; Jonathan, Philip; Randell, David. Bayes linear analysis of risks in sequential optimal design problems. Electron. J. Statist. 12 (2018), no. 2, 4002--4031. doi:10.1214/18-EJS1496. https://projecteuclid.org/euclid.ejs/1544518835

Export citation


  • R.J. Adler., The Geometry of Random Fields. Wiley, 1981.
  • J.O. Berger., Statistical Decision Theory. Springer, 1980.
  • K. Chaloner and I. Verdinelli. Bayesian experimental design: A review., Statistical Science, 10: 273–304, 1995.
  • M.H. DeGroot., Optimal Statistical Decisions. Wiley, 1970.
  • C.C. Drovandi, J.M. McGree, and A.N. Pettitt. Sequential Monte Carlo for Bayesian sequentially designed experiments for discrete data., Comput. Stat. Data Anal., 57: 320–335, 2013.
  • C.C. Drovandi, J.M. McGree, and A.N. Pettitt. A Sequential Monte Carlo Algorithm to Incorporate Model Uncertainty in Bayesian Sequential Design., Journal of Computational and Graphical Statistics, 23: 3–24, 2014. doi: 10.1080/10618600.2012.730083.
  • I. Ford, D.M. Titterington, and C.P. Kitsos. Recent advances in nonlinear experimental design., Technometrics, 31 (1): 49–60, 1989. ISSN 00401706.. URL http://www.jstor.org/stable/1270364.
  • M. Goldstein and J. Rougier. Reified bayesian modelling and inference for physical systems., Journal of Statistical Planning and Inference, 139: 1221–1239, 2009. doi: 10.1016/j.jspi.2008.07.019.
  • M. Goldstein and A. Seheult. Prior Viability Assessment for Bayesian Analysis., Journal of Statistical Planning and Inference, 138 (5): 1271–1286, 2008. ISSN 03783758. doi: 10.1016/j.jspi.2007.04.023.
  • P. Hennig and C.J. Schuler. Entropy search for information-efficient global optimization., Journal of Machine Learning Research, 13: 1809–1837, 2012.
  • B. Hirst, P. Jonathan, F. Gonzalez del Cueto, D. Randell, and O. Kosut. Locating and quantifying gas emission sources using remotely obtained concentration data., Atmospheric Environment, 45: 141–158, 2012.
  • X. Huan and Y. Marzouk. Sequential bayesian optimal experimental design via approximate dynamic programming., Forthcoming, 2016. URL https://arxiv.org/abs/1604.08320.
  • D.R. Jones, M. Schonlau, and W.J. Welch. Efficient global optimization of expensive black-box functions., Journal of Global Optimization, 13: 455–492, 1998.
  • M. Jones, M. Goldstein, P. Jonathan, and D. Randell. Supplementary Material: Bayes Linear Analysis of Risks in Sequential Optimal Design Problems. doi:, 10.1214/18-EJS1496SUPP.
  • M. Jones, M. Goldstein, D. Randell, and P. Jonathan. Bayes linear analysis for bayesian optimal design., Journal of Statistical Planning and Inference, 171: 115–129, 2016. doi: 10.1016/j.jspi.2015.10.011.
  • M.C. Kennedy and A. O’Hagan. Bayesian calibration of computer models., Journal of the Royal Statistical Society, Series B (Statistical Methodology), 63: 425–464, 2001.
  • D.V. Lindley., Bayesian Statistics, A Review. Society for Industrial and Applied Mathematics, 1972.
  • P. Muller, D.A. Berry, A.P. Grieve, M. Smith, and M. Krams. Simulation based sequential bayesian design., Journal of Statistical Planning and Inference, 137: 3140–3150, 2007. doi: 10.1016/j.jspi.2006.05.021.
  • A. O’Hagan. Bayes-hermite quadrature., Journal of Statistical Planning and Inference, 29: 245–260, 1991.
  • F. Pasquill. Atmospheric dispersion of pollution., Quarterly Journal of the Royal Meteorological Society, 97: 369–395, 1971.
  • D. Randell, M. Goldstein, G. Hardman, and P. Jonathan. Bayesian linear inspection planning for large-scale physical systems., Proceedings of the Institution of Mechanical Engineers part O: Journal of Risk and Reliability, 224: 333–345, 2010. doi: 10.1243/1748006XJRR322.
  • C.E. Rasmussen and Z. Ghahramani. Bayesian monte carlo. In, Advances in Neural Information Processing Systems. MIT Press, 2002.
  • J.Q. Smith., Bayesian Decision Analysis: Principles and Practice. Cambridge University Press, 2010.
  • J.M. Stockie. The mathematics of atmospheric dispersion modelling., SIAM Review, 53: 349–372, 2011.
  • I. Vernon, M. Goldstein, and R.G. Bower. Galaxy formation: a bayesian uncertainty analysis., Bayesian Analysis, 5: 619–669, 2010. doi: 10.1214/10-BA524.
  • D. Williamson and M. Goldstein. Bayesian policy support for adaptive strategies using computer models for complex physical systems., Journal of the Operational Research Society, 63: 1021–1033, 2012. doi: 10.1057/jors.2011.110.
  • D. Williamson, M. Goldstein, L. Allison, A. Blaker, P. Challenor, L. Jackson, and M. Yamazaki. History Matching for Exploring and Reducing Climate Model Parameter Space Using Observations and a Large Perturbed Physics Ensemble., Climate Dynamics, 41 (7-8): 1703–1729, 2013. ISSN 09307575. doi: 10.1007/s00382-013-1896-4.

Supplemental materials