The Annals of Applied Statistics

Controlled stratification for quantile estimation

Claire Cannamela, Josselin Garnier, and Bertrand Iooss

Full-text: Open access


In this paper we propose and discuss variance reduction techniques for the estimation of quantiles of the output of a complex model with random input parameters. These techniques are based on the use of a reduced model, such as a metamodel or a response surface. The reduced model can be used as a control variate; or a rejection method can be implemented to sample the realizations of the input parameters in prescribed relevant strata; or the reduced model can be used to determine a good biased distribution of the input parameters for the implementation of an importance sampling strategy. The different strategies are analyzed and the asymptotic variances are computed, which shows the benefit of an adaptive controlled stratification method. This method is finally applied to a real example (computation of the peak cladding temperature during a large-break loss of coolant accident in a nuclear reactor).

Article information

Ann. Appl. Stat., Volume 2, Number 4 (2008), 1554-1580.

First available in Project Euclid: 8 January 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Quantile estimation Monte Carlo methods variance reduction computer experiments


Cannamela, Claire; Garnier, Josselin; Iooss, Bertrand. Controlled stratification for quantile estimation. Ann. Appl. Stat. 2 (2008), no. 4, 1554--1580. doi:10.1214/08-AOAS186.

Export citation


  • Avramidis, A. N. and Wilson, J. R. (1988). Correlation-induction techniques for estimating quantiles in simulation experiments. Oper. Res. 46 574–591.
  • David, H. A. (1981). Order Statistics. Wiley, New York.
  • Davidson, R. and MacKinnon, J. G. (1992). Regression-based methods for using control variates in Monte Carlo experiments. J. Econometrics 54 203–222.
  • Dielman, T., Lowry, C. and Pfaffenberger, R. (1994). A comparison of quantile estimators. Communications in Statistics—Simulation and Computation 23 355–371.
  • Fang, K.-T., Li, R. and Sudjianto, A. (2006). Design and Modeling for Computer Experiments. Chapman and Hall, London.
  • Fishman, G. S. (1996). Monte Carlo Concepts, Algorithms and Applications. Springer, New York.
  • Glasserman, P., Heidelberger, P. and Shahabuddin, P. (1998). Stratification issues in estimating Value-at-Risk. In Proceedings of the 1999 Winter Simulation Conference 351–359. IEEE Computer Society Press, Piscataway, New Jersey.
  • Glynn, P. (1996). Importance sampling for Monte Carlo estimation of quantiles. In Mathematical Methods in Stochastic Simulation and Experimental Design: Proceedings of the 2nd St. Petersburg Workshop on Simulation 180–185. Publishing House of Saint Petersburg University.
  • Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. Chapman and Hall, London.
  • Hastie, T. J., Tibshirani, R. J. and Friedman, J. H. (2001). The Elements of Statistical Learning: Data Mining, Inference and Prediction. Springer, New York.
  • Hesterberg, T. C. (1993). Control variates and importance sampling for the bootstrap. In Proceedings of the Statistical Computing Section of the American Statistical Association 40–48.
  • Hesterberg, T. C. (1995). Average importance sampling and defensive mixture distributions. Technometrics 37 185–194.
  • Hesterberg, T. C. and Nelson, B. L. (1998). Control variates for probability and quantile estimation. Management Science 44 1295–1312.
  • Hsu, J. C. and Nelson, B. L. (1987). Control variates for quantile estimation. In Proceedings of the 1987 Winter Simulation Conference (A. Thesen, H. Grant and W. D. Kelton, eds.) 434–444.
  • Jones, D., Schonlau, M. and Welch, W. (1998). Efficient global optimization of expensive black-box functions. Journal of Global Optimization 13 455–492.
  • Law, A. M. and Kelton, W. D. (1991). Simulation Modeling and Analysis, 2nd ed. McGraw-Hill, New York.
  • Lin, D. K. J. (1993). A new class of supersaturated design. Technometrics 35 28–31.
  • Marrel, A., Iooss, B., Van Dorpe, F. and Volkova, E. (2008). An efficient methodology for modeling complex computer codes with Gaussian processes. Comput. Statist. Data Anal. DOI: 10.1016/j.csda.2008.03.026. In press.
  • Nelson, B. L. (1990). Control variates remedies. Oper. Res. 38 974–992.
  • Nutt, W. T. and Wallis, G. B. (2004). Evaluation of nuclear safety from the outputs of computer codes in the presence of uncertainties. Reliab. Eng. Syst. Safety 83 57–77.
  • Oakley, J. (2004). Estimating percentiles of uncertain computer code outputs. Appl. Statist. 53 83–93.
  • Oh, M. S. and Berger, J. O. (1992). Adaptive importance sampling in Monte Carlo integration. J. Stat. Comput. Simul. 41 143–168.
  • Petruzzi, A., D’Auria, F., Micaelli, J.-C., De Crecy, A. and Royen, J. (2004). The BEMUSE programme (Best-Estimate Methods—Uncertainty and Sensitivity Evaluation). In Proceedings of the Int. Meet. on Best-Estimate Methods in Nuclear Installation Safety Analysis (BE-2004) IX 1 225–235. Washington, USA.
  • Ranjan, P., Bingham, D. and Michailidis, G. (2008). Sequential experiment design for contour estimation from complex computer codes. Technometrics. To appear.
  • Rao, C. R. (1973). Linear Statistical Inference and Its Applications. Wiley, New York.
  • Rubinstein, R. Y. (1981). Simulation and the Monte Carlo Method. Wiley, New York.
  • Rutherford, B. (2006). A response-modeling alternative to surrogate models for support in computational analyses. Reliab. Eng. Syst. Safety 91 1322–1330.
  • Sacks, J., Welch, W. J., Mitchell, T. J. and Wynn, H. P. (1989). Design and analysis of computer experiments. Statist. Sci. 4 409–435.
  • Schonlau, M. and Welch, W. J. (2005). Screening the input variables to a computer model via analysis of variance and visualization. In Screening Methods for Experimentation and Industry, Drug Discovery and Genetics (A. M. Dean and S. M. Lewis, eds.) 308–327. Springer, Berlin.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. J. Roy. Statist. Soc. Ser B 58 267–288.
  • Vazquez, E. and Piera Martinez, M. (2008). Estimation of the volume of an excursion set of a Gaussian process using intrinsic kriging. J. Statist. Plann. Inference. To appear. Available at arXiv:math/0611273.
  • Volkova, E., Iooss, B. and Van Dorpe, F. (2008). Global sensitivity analysis for a numerical model of radionuclide migration, from the RRC “Kurchatov Institute” radwaste disposal site. Stoch. Environ. Res. Risk Assess. 22 17–31.