Annals of Applied Statistics

Incorporating external information in analyses of clinical trials with binary outcomes

Minge Xie, Regina Y. Liu, C. V. Damaraju, and William H. Olson

Full-text: Open access


External information, such as prior information or expert opinions, can play an important role in the design, analysis and interpretation of clinical trials. However, little attention has been devoted thus far to incorporating external information in clinical trials with binary outcomes, perhaps due to the perception that binary outcomes can be treated as normally-distributed outcomes by using normal approximations. In this paper we show that these two types of clinical trials could behave differently, and that special care is needed for the analysis of clinical trials with binary outcomes. In particular, we first examine a simple but commonly used univariate Bayesian approach and observe a technical flaw. We then study the full Bayesian approach using different beta priors and a new frequentist approach based on the notion of confidence distribution (CD). These approaches are illustrated and compared using data from clinical studies and simulations. The full Bayesian approach is theoretically sound, but surprisingly, under skewed prior distributions, the estimate derived from the marginal posterior distribution may not fall between those from the marginal prior and the likelihood of clinical trial data. This counterintuitive phenomenon, which we call the “discrepant posterior phenomenon,” does not occur in the CD approach. The CD approach is also computationally simpler and can be applied directly to any prior distribution, symmetric or skewed.

Article information

Ann. Appl. Stat., Volume 7, Number 1 (2013), 342-368.

First available in Project Euclid: 9 April 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Bayesian method combining information confidence distribution expert opinion requentist method discrepant posterior prior distribution


Xie, Minge; Liu, Regina Y.; Damaraju, C. V.; Olson, William H. Incorporating external information in analyses of clinical trials with binary outcomes. Ann. Appl. Stat. 7 (2013), no. 1, 342--368. doi:10.1214/12-AOAS585.

Export citation


  • Berger, J. (2006). The case for objective Bayesian analysis. Bayesian Anal. 1 385–402 (electronic).
  • Berry, D. A. and Stangl, D. (1996). Bayesian Biostatistics. Dekker, New York.
  • Bickel, D. R. (2006). Incorporating expert knowledge into frequentist inference by combining generalized confidence distributions. Unpublished manuscript.
  • Boos, D. D. and Monahan, J. F. (1986). Bootstrap methods using prior information. Biometrika 73 77–83.
  • Brandes, J. L., Saper, J. R., Diamond, M., Couch, J. R., Lewis, D. W., Schmitt, J., Neto, W., Schwabe, S. and Jacobs, D. (2004). Topiramate for migraine prevention: A randomized controlled trial. JAMA 291 965–973.
  • Carlin, B. P. and Louis, T. A. (2009). Bayesian Methods for Data Analysis, 3rd ed. CRC Press, Boca Raton, FL.
  • Dawid, A. P., Stone, M. and Zidek, J. V. (1973). Marginalization paradoxes in Bayesian and structural inference. J. Roy. Statist. Soc. Ser. B 35 189–233.
  • Efron, B. (1986). Why isn’t everyone a Bayesian? Amer. Statist. 40 1–11.
  • Efron, B. (1998). R. A. Fisher in the 21st century (invited paper presented at the 1996 R. A. Fisher Lecture). Statist. Sci. 13 95–122.
  • Fisher, R. A. (1930). Inverse probability. Proc. Cambridge Philos. Soc. 26 528–535.
  • Fraser, D. A. S. (1991). Statistical inference: Likelihood to significance. J. Amer. Statist. Assoc. 86 258–265.
  • Gelman, A., Carlin, J. B., Stern, H. S. and Rubin, D. B. (2004). Bayesian Data Analysis, 2nd ed. Chapman & Hall/CRC, Boca Raton, FL.
  • Genest, C. and Zidek, J. V. (1986). Combining probability distributions: A critique and an annotated bibliography. Statist. Sci. 1 114–148.
  • Joseph, L., du Berger, R. and Bélisle, P. (1997). Bayesian and mixed Bayesian/likelihood criteria for sample size determination. Stat. Med. 16 769–781.
  • Lawless, J. F. and Fredette, M. (2005). Frequentist prediction intervals and predictive distributions. Biometrika 92 529–542.
  • Lee, P. M. (1992). Bayesian Statistics: An Introduction, 3rd ed. Wiley, New York.
  • Lehmann, E. L. (1993). The Fisher, Neyman–Pearson theories of testing hypotheses: One theory or two? J. Amer. Statist. Assoc. 88 1242–1249.
  • Neyman, J. (1941). Fiducial argument and the theory of confidence intervals. Biometrika 32 128–150.
  • Olkin, I. and Liu, R. (2003). A bivariate beta distribution. Statist. Probab. Lett. 62 407–412.
  • Parmar, M. K. B., Spiegelhalter, D. J. and Freedman, L. S. (1994). The chart trials: Bayesian design and monitoring in practice. Stat. Med. 13 1297–1312.
  • Schweder, T. and Hjort, N. L. (2002). Confidence and likelihood. Scand. J. Statist. 29 309–332.
  • Schweder, T. and Hjort, N. L. (2003). Frequentist analogues of priors and posteriors. In Econometrics and the Philosophy of Economics (B. P. Stigum, ed.) 285–317. Princeton Univ. Press, Princeton, NJ.
  • Schweder, T. and Hjort, N. L. (2012). Confidence, Likelihood and Probability. Cambridge Univ. Press, Cambridge.
  • Silberstein, S. D., Neto, W., Schmitt, J., Jacobs, D. and MIGR-001 Study Group (2004). Topiramate in migraine prevention: Results of a large controlled trial. Arch. Neurol. 61 490–495.
  • Singh, K., Xie, M. and Strawderman, W. E. (2001). Confidence distributions—concept, theory and applications. Technical Report, Dept. Statistics, Rutgers Univ., Piscataway, NJ.
  • Singh, K., Xie, M. and Strawderman, W. E. (2005). Combining information from independent sources through confidence distributions. Ann. Statist. 33 159–183.
  • Singh, K., Xie, M. and Strawderman, W. E. (2007). Confidence distribution (CD)—distribution estimator of a parameter. In Complex Datasets and Inverse Problems (R. Liu et al., eds.). Institute of Mathematical Statistics Lecture Notes—Monograph Series 54 132–150. IMS, Beachwood, OH.
  • Singh, K. and Xie, M. (2012). CD posterior—combining prior and data through confidence distributions. In Contemporary Developments in Bayesian Analysis and Statistical Decision Theory: A Festschrift in Honor of William E. Strawderman (D. Fourdrinier et al., eds.). IMS Collection 8 200–214. IMS, Beachwood, OH.
  • Spiegelhalter, D. J., Abrams, K. R. and Myles, J. P. (2004). Bayesian Approaches to Clinical Trials and Health-Care Evaluation. Wiley, New York.
  • Spiegelhalter, D. J., Freedman, L. S. and Parmar, M. K. B. (1994). Bayesian approaches to randomized trials. J. Roy. Statist. Soc. Ser. A 157 357–416.
  • Tian, L., Wang, R., Cai, T. and Wei, L.-J. (2011). The highest confidence density region and its usage for joint inferences about constrained parameters. Biometrics 67 604–610.
  • Wasserman, L. (2007). Why isn’t everyone a Bayesian. In The Science of Bradley Efron (C. N. Morris and R. Tibshirani, eds.) 260–261. Springer, New York.
  • Wijeysundera, D. N., Austin, P. C., Hux, J. E., Beattie, W. S. and Laupacis, A. (2009). Bayesian statistical inference enhances the interpretation of contemporary randomized controlled trials. J. Clin. Epidemiol. 62 13–21.e5.
  • Xie, M., Singh, K. and Strawderman, W. E. (2011). Confidence distributions and a unifying framework for meta-analysis. J. Amer. Statist. Assoc. 106 320–333.
  • Xie, M. and Singh, K. (2012). Confidence distribution, the frequentist distribution estimator of a parameter—a review (with discussion). Internat. Statist. Rev. To appear.
  • Xie, M., Liu, R. Y., Damaraju, C. V. and Olson, W. H. (2013). Supplement to “Incorporating external information in analyses of clinical trials with binary outcomes.” DOI:10.1214/12-AOAS585SUPP.

Supplemental materials

  • Supplementary material: Appendix: MCMC algorithm and CD examples. Appendix I contains a Metropolis–Hastings algorithm used in Section 2. Appendix II presents two CD examples that are relevant to the exposition of this paper.