Bayesian Analysis

A Bayesian Nonparametric Multiple Testing Procedure for Comparing Several Treatments Against a Control

Luis Gutiérrez, Andrés F. Barrientos, Jorge González, and Daniel Taylor-Rodríguez

Full-text: Open access

Abstract

We propose a Bayesian nonparametric strategy to test for differences between a control group and several treatment regimes. Most of the existing tests for this type of comparison are based on the differences between location parameters. In contrast, our approach identifies differences across the entire distribution, avoids strong modeling assumptions over the distributions for each treatment, and accounts for multiple testing through the prior distribution on the space of hypotheses. The proposal is compared to other commonly used hypothesis testing procedures under simulated scenarios. Two real applications are also analyzed with the proposed methodology.

Article information

Source
Bayesian Anal., Volume 14, Number 2 (2019), 649-675.

Dates
First available in Project Euclid: 18 September 2018

Permanent link to this document
https://projecteuclid.org/euclid.ba/1537258138

Digital Object Identifier
doi:10.1214/18-BA1122

Mathematical Reviews number (MathSciNet)
MR3959876

Zentralblatt MATH identifier
07089621

Subjects
Primary: 62G10: Hypothesis testing 62G07: Density estimation
Secondary: 62G05: Estimation

Keywords
Bayes factor Dependent Dirichlet process spike and slab priors shift function

Rights
Creative Commons Attribution 4.0 International License.

Citation

Gutiérrez, Luis; Barrientos, Andrés F.; González, Jorge; Taylor-Rodríguez, Daniel. A Bayesian Nonparametric Multiple Testing Procedure for Comparing Several Treatments Against a Control. Bayesian Anal. 14 (2019), no. 2, 649--675. doi:10.1214/18-BA1122. https://projecteuclid.org/euclid.ba/1537258138


Export citation

References

  • Agencia de Calidad de la Educación (2017). “SIMCE.” http://www.agenciaeducacion.cl/evaluaciones/que-es-el-simce/.
  • Altomare, D., Consonni, G., and La Rocca, L. (2013). “Objective Bayesian search of Gaussian directed acyclic graphical models for ordered variables with non-local priors.” Biometrics, 69(2): 478–87.
  • Barrientos, A. F., Jara, A., and Quintana, F. A. (2012). “On the support of MacEachern’s dependent Dirichlet processes and extensions.” Bayesian Analysis, 7: 277–310.
  • Basu, S. and Chib, S. (2003). “Marginal likelihood and Bayes factors for Dirichlet Process Mixture models.” Journal of the American Statistical Association, 98: 224–235.
  • Berger, J. and Pericchi, L. (1996). “The intrinsic Bayes factor for model selection and prediction.” Journal of the American Statistical Association, 91(433): 109–122.
  • Bhattacharya, A. and Dunson, D. (2012). “Nonparametric Bayes classification and hypothesis testing on manifolds.” Journal of Multivariate Analysis, 111: 1–19.
  • Canale, A. and Scarpa, B. (2016). “Bayesian nonparametric location-scale-shape mixtures.” Test, 25: 113–130.
  • Castillo, I., Schmidt-Hieber, J., Van der Vaart, A., et al. (2015). “Bayesian linear regression with sparse priors.” The Annals of Statistics, 43(5): 1986–2018.
  • Chen, Y. and Hanson, T. E. (2014). “Bayesian nonparametric k–sample tests for censored and uncensored data.” Computational Statistics and Data Analysis, 71: 335–346.
  • Cipolli, W., Hanson, T., and McLain, A. (2016). “Bayesian nonparametric multiple testing.” Computational Statistics and Data Analysis, 101: 64–79.
  • De Boeck, P. and Wilson, M. (2004). Explanatory item response models. A generalized linear and nonlinear approach. New York, USA: Springer.
  • De Iorio, M., Müller, P., Rosner, G. L., and MacEachern, S. N. (2004). “An ANOVA model for dependent random measures.” Journal of the American Statistical Association, 99: 205–215.
  • Doksum, K. (1974). “Empirical probability plots and statistical inference for nonlinear models in the two-sample case.” The Annals of Statistics, 2: 267–277.
  • Doksum, K. and Sievers, G. (1976). “Plotting with confidence: Graphical comparisons of two populations.” Biometrika, 63: 421–434.
  • Dunnett, C. W. (1955). “A multiple comparison procedure for comparing several treatments with a control.” Journal of the American Statistical Association, 50(272): 1096–1121.
  • Dunnett, C. W. and Tamhane, A. C. (1991). “Step-down multiple tests for comparing treatments with a control in unbalanced one-way layouts.” Statistics in Medicine, 10(6): 939–947.
  • Escobar, M. D. and West, M. (1995). “Bayesian density estimation and inference using mixtures.” Journal of the American Statistical Association, 90: 577–588.
  • Gao, X., Alvo, M., Chen, J., and Li, G. (2008). “Nonparametric multiple comparison procedures for unbalanced one-way factorial designs.” Journal of Statistical Planning and Inference, 138(8): 2574–2591.
  • George, E. I. and McCulloch, R. E. (1993). “Variable selection via Gibbs sampling.” Journal of the American Statistical Association, 88(423): 881–889.
  • Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian nonparametrics. New York, USA: Springer.
  • Gutiérrez, L., Barrientos, A. F., González, J., and Taylor-Rodríguez, D. (2018). “Supplementary Material for ‘A Bayesian nonparametric multiple testing procedure for comparing several treatments against a control’.” Bayesian Analysis.
  • Hjort, N. L., Holmes, C., Müller, P., and Walker, S. (2010). Bayesian nonparametrics. Cambridge, UK: Cambridge University Press.
  • Hochberg, Y. and Tamhane, A. (1987). Multiple Comparison Procedures. Wiley, New York.
  • Hollander, M. and Korwar, R. (1980). Nonparametric Bayesian Estimation of the Horizontal Distance Between Two Populations. Defense Technical Information Center.
  • Hollander, M. and Wolfe, D. (1999). Nonparametric Statistical Methods. Wiley Series in Probability and Statistics.
  • Holm, S. (1979). “A Simple Sequentially Rejective Multiple Test Procedure.” Scandinavian Journal of Statistics, 6(2): 65–70.
  • Holmes, C., Caron, F., Griffin, J., and Stephens, D. (2015). “Two–sample Bayesian nonparametric hypothesis testing.” Bayesian Analysis, 10(2): 297–320.
  • Ishwaran, H. and Rao, J. S. (2005). “Spike and slab variable selection: Frequentist and Bayesian strategies.” The Annals of Statistics, 33(2): 730–773.
  • Jeffreys, H. (1935). “Some Tests of Significance, Treated by the Theory of Probability.” Proceedings of the Cambridge Philosophy Society, 31: 203–222.
  • Jeffreys, H. (1961). The theory of probability (3rd. Ed.). Oxford, UK: Oxford University Press.
  • Jeffreys, W. H. and Berger, J. O. (1992). “Ockham’s razor and Bayesian analysis.” American Scientist, 80(1): 64–72.
  • Johnson, V. E. and Rossell, D. (2010). “On the use of non-local prior densities in Bayesian hypothesis tests.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(2): 143–170.
  • Johnson, V. E. and Rossell, D. (2012). “Bayesian Model Selection in High-Dimensional Settings.” Journal of the American Statistical Association, 107(498): 649–660.
  • Kalli, M., Griffin, J. E., and Walker, S. (2011). “Slice sampling mixture models.” Statistics and Computing, 21: 93–105.
  • Kass, R. E. and Raftery, A. E. (1995). “Bayes factors.” Journal of the American Statistical Association, 90: 773–795.
  • Kim, S., Dahl, D. B. D., and Vannucci, M. (2009). “Spiked Dirichlet process prior for Bayesian multiple hypothesis testing in random effects models.” Bayesian Analysis, 4(4): 707–732.
  • Labadi, L. A., Masuadi, E., and Zarepour, M. (2014). “Two-sample Bayesian nonparametric goodness-of-fit test.” arXiv preprint arXiv:1411.3427.
  • Ley, E. and Steel, M. F. (2009). “On the effect of prior assumptions in Bayesian model averaging with applications to growth regression.” Journal of applied econometrics, 24(4): 651–674.
  • Liang, F., Paulo, R., Molina, G., Clyde, M. a., and Berger, J. O. (2008). “Mixtures of g Priors for Bayesian Variable Selection.” Journal of the American Statistical Association, 103(481): 410–423.
  • Lu, H. H. S., Wells, M. T., and Tiwari, R. C. (1994). “Inference for Shift Functions in the Two-Sample Problem With Right-Censored Data: With Applications.” Journal of the American Statistical Association, 89(427): 1017–1026.
  • Ma, L. and Wong, W. H. (2011). “Coupling optional Pólya trees and the two sample problem.” Journal of the American Statistical Association, 106: 1553–1565.
  • MacEachern, S. N. (1999). “Dependent nonparametric processes.” In ASA Proceedings of the Section on Bayesian Statistical Science, Alexandria, VA. American Statistical Association.
  • MacEachern, S. N. (2000). “Dependent Dirichlet processes.” Technical report, Department of Statistics, The Ohio State University.
  • MIDE UC (2017). “SEPA.” http://www.sepauc.cl/.
  • Mitchell, T. J. and Beauchamp, J. J. (1988). “Bayesian variable selection in linear regression.” Journal of the American Statistical Association, 83(404): 1023–1032.
  • Müller, P. and Mitra, R. (2013). “Bayesian Nonparametric Inference–Why and How.” Bayesian Analysis, 8(2): 269–302.
  • Müller, P. and Quintana, F. (2004). “Nonparametric Bayesian data analysis.” Statistical Science, 19: 95–110.
  • Müller, P., Quintana, F. A., and Rosner, G. (2004). “A method for combining inference across related nonparametric Bayesian models.” Journal of the Royal Statistical Society, Series B, 66: 735–749.
  • OECD (2016). “PISA 2015 Results (Volume I).” URL http://www.oecd.org/education/pisa-2015-results-volume-i-9789264266490-en.htm
  • Pati, D., Dunson, D. B., and Tokdar, S. T. (2013). “Posterior consistency in conditional distribution estimation.” Journal of Multivariate Analysis, 116: 456–472.
  • Ramanan, G. and Berry, D. (1998). “A Bayesian multiple comparisons using Dirichlet process priors.” Journal of the American Statistical Association, 93(443): 1130–1139.
  • Ročková, V. and George, E. I. (2014). “EMVS: The EM approach to Bayesian variable selection.” Journal of the American Statistical Association, 109(506): 828–846.
  • Ročková, V. and George, E. I. (2016). “The Spike-and-Slab LASSO.” Journal of the American Statistical Association.
  • Scott, J. G. (2009). “Nonparametric Bayesian multiple testing for longitudinal performance stratification.” The Annals of Applied Statistics, 3(4): 1655–1674.
  • Scott, J. G. and Berger, J. O. (2006). “An Exploration of aspects of Bayesian multiple testing.” Journal of Statistical Planning and Inference, 136(7): 2144–2162.
  • Scott, J. G. and Berger, J. O. (2010). “Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem.” The Annals of Statistics, 38(5): 2587–2619.
  • Sethuraman, J. (1994). “A constructive definition of Dirichlet priors.” Statistica Sinica, 2: 639–650.
  • Simovici, D. A. and Djeraba, C. (2008). Partially Ordered Sets, 129–172. London: Springer London.
  • Soriano, J. and Ma, L. (2017). “Probabilistic multi–resolution scanning for two–sample differences.” Journal of the Royal Statistical Society: Series B, 79: 547–572.
  • van der Linden, W. J. (ed.) (2016). Handbook of item response theory. Three volume set. Boca Raton, FL: Chapman and Hall/CRC.
  • Walker, S. (2007). “Sampling the Dirichlet mixture model with Slices.” Communications in Statistics- Simulation and Computation, 36: 45–54.
  • Wells, M. T. and Tiwari, R. C. (1989). “Bayesian quantile plots and statistical inference for nonlinear models in the two sample case with incomplete data.” Communications in Statistics – Theory and Methods, 18(8): 2955–2964.
  • Wilson, M., Iversen, E., Clyde, M. A., Schmidler, S. C., and Schildkraut, J. M. (2010). “Bayesian model search and multilevel inference for SNP association studies.” The Annals of Applied Statistics, 4(3): 1342–1364.
  • Womack, A. J., Fuentes, C., and Taylor-Rodriguez, D. (2015). “Model Space Priors for Objective Sparse Bayesian Regression.” arXiv preprint arXiv:1511.04745.
  • Womack, A. J., León-Novelo, L., and Casella, G. (2014). “Inference From Intrinsic Bayes’ Procedures Under Model Selection and Uncertainty.” Journal of the American Statistical Association, 109(507): 1040–1053.
  • Zellner, A. and Siow, A. (1980). “Posterior odds ratios for selected regression hypotheses.” In Bayesian Statistics: Proceedings of the First International Meeting Held in Valencia, 585–603. URL http://www.springerlink.com/index/5300770UP12246M9.pdf

Supplemental materials

  • Supplementary Material for ‘A Bayesian nonparametric multiple testing procedure for comparing several treatments against a control’. The online Supplementary Material contains the Gibbs Algorithm described in Section 3.4, as well as the image plots of the comparison between our proposal and other classical hypothesis tests (Section 4.2), including both multiple and two-sample cases.