Statistical Science

Models for the Assessment of Treatment Improvement: The Ideal and the Feasible

P. C. Álvarez-Esteban, E. del Barrio, J. A. Cuesta-Albertos, and C. Matrán

Full-text: Open access


Comparisons of different treatments or production processes are the goals of a significant fraction of applied research. Unsurprisingly, two-sample problems play a main role in statistics through natural questions such as “Is the the new treatment significantly better than the old?” However, this is only partially answered by some of the usual statistical tools for this task. More importantly, often practitioners are not aware of the real meaning behind these statistical procedures. We analyze these troubles from the point of view of the order between distributions, the stochastic order, showing evidence of the limitations of the usual approaches, paying special attention to the classical comparison of means under the normal model. We discuss the unfeasibility of statistically proving stochastic dominance, but show that it is possible, instead, to gather statistical evidence to conclude that slightly relaxed versions of stochastic dominance hold.

Article information

Statist. Sci., Volume 32, Number 3 (2017), 469-485.

First available in Project Euclid: 1 September 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Stochastic dominance similarity two-sample comparison trimmed distributions winsorized distributions Behrens–Fisher problem index of stochastic dominance


Álvarez-Esteban, P. C.; del Barrio, E.; Cuesta-Albertos, J. A.; Matrán, C. Models for the Assessment of Treatment Improvement: The Ideal and the Feasible. Statist. Sci. 32 (2017), no. 3, 469--485. doi:10.1214/17-STS616.

Export citation


  • Álvarez-Esteban, P. C., del Barrio, E., Cuesta-Albertos, J. A. and Matrán, C. (2008). Trimmed comparison of distributions. J. Amer. Statist. Assoc. 103 697–704.
  • Álvarez-Esteban, P. C., del Barrio, E., Cuesta-Albertos, J. A. and Matrán, C. (2012). Similarity of samples and trimming. Bernoulli 18 606–634.
  • Álvarez-Esteban, P. C., del Barrio, E., Cuesta-Albertos, J. A. and Matrán, C. (2014). A contamination model for approximate stochastic order: Extended version. Available at
  • Álvarez-Esteban, P. C., del Barrio, E., Cuesta-Albertos, J. A. and Matrán, C. (2016). A contamination model for the stochastic order. TEST 25 751–774.
  • Anderson, G. (1996). Nonparametric tests for stochastic dominance. Econometrica 64 1183–1193.
  • Barrett, G. F. and Donald, S. G. (2003). Consistent tests for stochastic dominance. Econometrica 71 71–104.
  • Berger, R. L. (1988). A nonparametric, intersection-union test for stochastic order. In Statistical Decision Theory and Related Topics, IV, Vol. 2 (West Lafayette, Ind., 1986) 253–264. Springer, New York.
  • Chen, G., Chen, J. and Chen, Y. (2002). Statistical inference on comparing two distribution functions with a possible crossing point. Statist. Probab. Lett. 60 329–341.
  • Davidson, R. and Duclos, J.-Y. (2000). Statistical inference for stochastic dominance and for the measurement of poverty and inequality. Econometrica 68 1435–1464.
  • Davidson, R. and Duclos, J.-Y. (2013). Testing for restricted stochastic dominance. Econometric Rev. 32 84–125.
  • del Barrio, E., Giné, E. and Matrán, C. (1999). Central limit theorems for the Wasserstein distance between the empirical and the true distributions. Ann. Probab. 27 1009–1071.
  • Dette, H. and Munk, A. (1998). Validation of linear regression models. Ann. Statist. 26 778–800.
  • Hawkins, D. L. and Kochar, S. C. (1991). Inference for the crossing point of two continuous CDFs. Ann. Statist. 19 1626–1638.
  • Hodges, J. L. Jr. and Lehmann, E. L. (1954). Testing the approximate validity of statistical hypotheses. J. Roy. Statist. Soc. Ser. B 16 261–268.
  • Lehmann, E. L. (1955). Ordered families of distributions. Ann. Math. Stat. 26 399–419.
  • Lehmann, E. L. and Rojo, J. (1992). Invariant directional orderings. Ann. Statist. 20 2100–2110.
  • Leshno, M. and Levy, H. (2002). Preferred by “All” and preferred by “Most” decision makers: Almost stochastic dominance. Manage. Sci. 48 1074–1085.
  • Linton, O., Maasoumi, E. and Whang, Y.-J. (2005). Consistent testing for stochastic dominance under general sampling schemes. Rev. Econ. Stud. 72 735–765.
  • Linton, O., Song, K. and Whang, Y.-J. (2010). An improved bootstrap test of stochastic dominance. J. Econometrics 154 186–202.
  • Liu, J. and Lindsay, B. G. (2009). Building and using semiparametric tolerance regions for parametric multinomial models. Ann. Statist. 37 3644–3659.
  • Mann, H. B. and Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18 50–60.
  • McFadden, D. (1989). Testing for stochastic dominance. In Studies in the Economics of Uncertainty: In Honor of Josef Hadar (T. B. Fomby and T. K. Seo, eds.) Springer, New York.
  • Müller, A. and Stoyan, D. (2002). Comparison Methods for Stochastic Models and Risks. Wiley, Chichester.
  • Munk, A. and Czado, C. (1998). Nonparametric validation of similar distributions and assessment of goodness of fit. J. R. Stat. Soc. Ser. B. Stat. Methodol. 60 223–241.
  • Raghavachari, M. (1973). Limiting distributions of Kolmogorov–Smirnov type statistics under the alternative. Ann. Statist. 1 67–73.
  • Rudas, T., Clogg, C. C. and Lindsay, B. G. (1994). A new index of fit based on mixture methods for the analysis of contingency tables. J. Roy. Statist. Soc. Ser. B 56 623–639.
  • Shaked, M. and Shanthikumar, J. G. (2007). Stochastic Orders. Springer, New York.
  • Tsetlin, I., Winkler, R. L., Huang, R. J. and Tzeng, L. Y. (2015). Generalized almost stochastic dominance. Oper. Res. 63 363–377.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York.