The Annals of Applied Statistics

Randomization inference for stepped-wedge cluster-randomized trials: An application to community-based health insurance

Xinyao Ji, Gunther Fink, Paul Jacob Robyn, and Dylan S. Small

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


National health insurance schemes are generally impractical in low-income countries due to limited resources and low organizational capacity. In response to such obstacles, community-based health insurance (CBHI) schemes have emerged over the past 20 years. CBHIs are designed to reduce the financial burden generated by unanticipated treatment cost among individuals falling sick, and thus are expected to make health care more affordable. In this paper, we investigate whether CBHI schemes effectively protect individuals against large financial shocks using a stepped-wedge cluster-randomized design on data from a CBHI program rolled out in rural Burkina Faso. We investigate statistical properties of the stepped-wedge design following the parametric mixed model approach proposed by Hussey and Hughes in 2007. We find that testing for the treatment effect is generally sensitive to specification of the parametric model. For instance, if we fail to account for cluster-by-time interactions present in the data, the Type I error rate is severely inflated. We develop a more robust and efficient strategy—randomization inference. We demonstrate how to apply randomization inference to test for constant treatment effects and discuss test statistics suitable for the stepped-wedge design. Randomization inference guarantees a valid Type I error rate; simulation studies show that randomization inference test statistics also have power that is comparable to the currently used procedures that do not guarantee a valid Type I error rate. Finally, we apply our proposed method to the Burkina Faso CBHI dataset. We conclude that the insurance had limited effects on reducing the likelihood of low to moderate levels of catastrophic health expenditure, but substantially reduced the likelihood of extremely high health expenditure that exceeds half of a person’s monthly income.

Article information

Ann. Appl. Stat., Volume 11, Number 1 (2017), 1-20.

Received: March 2016
Revised: July 2016
First available in Project Euclid: 8 April 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Randomization inference stepped-wedge cluster-randomized trials community-based health insurance


Ji, Xinyao; Fink, Gunther; Robyn, Paul Jacob; Small, Dylan S. Randomization inference for stepped-wedge cluster-randomized trials: An application to community-based health insurance. Ann. Appl. Stat. 11 (2017), no. 1, 1--20. doi:10.1214/16-AOAS969.

Export citation


  • Asenso-Okyere, W. K., Osei-Akoto, I., Anum, A. and Appiah, E. N. (1997). Willingness to pay for health insurance in a developing economy. A pilot study of the informal sector of Ghana using contingent valuation. Health Policy 42 223–237.
  • Bellan, S. E., Pulliam, J. R., Pearson, C. A., Champredon, D., Fox, S. J., Skrip, L., Galvani, A. P., Gambhir, M., Lopman, B. A., Porco, T. C. et al. (2015). Statistical power and validity of Ebola vaccine trials in Sierra Leone: A simulation study of trial design and analysis. Lancet, Infect. Dis. 15 703–710.
  • Braun, T. M. and Feng, Z. (2001). Optimal permutation tests for the analysis of group randomized trials. J. Amer. Statist. Assoc. 96 1424–1432.
  • Brown, C. A. and Lilford, R. J. (2006). The stepped wedge trial design: A systematic review. BMC Med. Res. Methodol. 6 54.
  • Devadasan, N., Ranson, K., Damme, W. V., Acharya, A. and Criel, B. (2006). The landscape of community health insurance in India: An overview based on 10 case studies. Health Policy 78 224–234.
  • De Allegri, M. D., Kouyaté, B., Becher, H., Gbangou, A., Pokhrel, S., Sanon, M. and Sauerborn, R. (2006). Understanding enrolment in community health insurance in sub-Saharan Africa: A population-based case-control study in rural Burkina Faso. Bull. World Health Organ. 84 852–858.
  • De Allegri, M. D., Pokhrel, S., Becher, H., Dong, H., Mansmann, U., Kouyaté, B., Kynast-Wolf, G., Gbangou, A., Sanon, M., Bridges, J. and Sauerborn, R. (2008). Step-wedge cluster-randomised community-based trials: An application to the study of the impact of community health insurance. Health Res. Policy Syst. 6 10.
  • Dimairo, M., Bradburn, M. and Walters, S. J. (2011). Sample size determination through power simulation; practical lessons from a stepped wedge cluster randomised trial (SW CRT). Trials 12 (Suppl 1) A26.
  • Ekman, B. (2004). Community-based health insurance in low-income countries: A systematic review of the evidence. Health Policy Plan. 19 249–270.
  • Fink, G., Robyn, P. J., Sié, A. and Sauerborn, R. (2013). Does health insurance improve health? Evidence from a randomized community-based insurance rollout in rural Burkina Faso. J. Health Econ. 32 1043–1056.
  • Fisher, R. A. (1935). The Design of Experiments. Oliver and Boyd, Edinburgh.
  • Gail, M. H., Mark, S. D., Carroll, R. J., Green, S. B. and Pee, D. (1996). On design considerations and randomization-based inference for community intervention trials. Stat. Med. 15 1069–1092.
  • Greene, W. H. (2003). Econometric Analysis. Pearson Education India.
  • Greevy, R., Silber, J. H., Cnaan, A. and Rosenbaum, P. R. (2004). Randomization inference with imperfect compliance in the ACE-inhibitor after anthracycline randomized trial. J. Amer. Statist. Assoc. 99 7–15.
  • Hall, A. J., Inskip, H. M., Loik, F., Day, N. E., O’Conor, G., Bosch, X. and Muir, C. S. (1987). The Gambia hepatitis intervention study. Cancer Res. 47 5782–5787.
  • Hansen, B. B. and Bowers, J. (2009). Attributing effects to a cluster-randomized get-out-the-vote campaign. J. Amer. Statist. Assoc. 104 873–885.
  • Ho, D. E. and Imai, K. (2006). Randomization inference with natural experiments: An analysis of ballot effects in the 2003 California recall election. J. Amer. Statist. Assoc. 101 888–900.
  • Hoaglin, D. C., Mosteller, F. and Tukey, J. W. (2000). Understanding Robust and Exploratory Data Analysis. Wiley, New York.
  • Hussey, M. A. and Hughes, J. P. (2007). Design and analysis of stepped wedge cluster randomized trials. Contemp. Clin. Trials 28 182–191.
  • Imbens, G. W. and Rubin, D. B. (2015). Causal Inference—for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge Univ. Press, New York.
  • Jacqmin-Gadda, H., Sibillot, S., Proust, C., Molina, J.-M. and Thiébaut, R. (2007). Robustness of the linear mixed model to misspecified error distribution. Comput. Statist. Data Anal. 51 5142–5154.
  • Mdege, N. D., Man, M. S., Taylor, C. A. and Torgerson, D. J. (2011). Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J. Clin. Epidemiol. 64 936–948.
  • Moulton, L. H., Golub, J. E., Durovni, B., Cavalcante, S. C., Pacheco, A. G., Saraceni, V., King, B. and Chaisson, R. E. (2007). Statistical design of THRio: A phased implementation clinic-randomized study of a tuberculosis preventive therapy intervention. Clin. Trials 4 190–199.
  • Neyman, J. (1990). On the application of probability theory to agricultural experiments. Statist. Sci. 5 463–464.
  • Raz, J. (1990). Testing for no effect when estimating a smooth function by nonparametric regression: A randomization approach. J. Amer. Statist. Assoc. 85 132–138.
  • Rhoda, D. A., Murray, D. M., Andridge, R. R., Pennell, M. L. and Hade, E. M. (2011). Studies with staggered starts: Multiple baseline designs and group-randomized trials. Am. J. Publ. Health 101 2164–2169.
  • Robyn, P. J., Fink, G., Sié, A. and Sauerborn, R. (2012). Health insurance and health-seeking behavior: Evidence from a randomized community-based insurance rollout in rural Burkina Faso. Soc. Sci. Med. 75 595–603.
  • Rosenbaum, P. R. (2002a). Covariance adjustment in randomized experiments and observational studies. Statist. Sci. 17 286–327.
  • Rosenbaum, P. R. (2002b). Observational Studies. Springer, New York.
  • Rosenbaum, P. R. (2007). Interference between units in randomized experiments. J. Amer. Statist. Assoc. 102 191–200.
  • Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and non-randomized studies. J. Educ. Psychol. 66 688–701.
  • Rubin, D. B. (1980). Randomization analysis of experimental data: The Fisher randomization test comment. J. Amer. Statist. Assoc. 75 591–593.
  • Sen, P. K. (1968). On a class of aligned rank order tests in two-way layouts. Ann. Math. Stat. 39 1115–1124.
  • Small, D. S., Ten Have, T. R. and Rosenbaum, P. R. (2008). Randomization inference in a group-randomized trial of treatments for depression: Covariate adjustment, noncompliance, and quantile effects. J. Amer. Statist. Assoc. 103 271–279.
  • Tukey, J. W. (1993). Tightening the clinical trial. Control. Clin. Trials 14 266–285.
  • van der Tweel, I. and van der Graaf, R. (2013). Issues in the use of stepped wedge cluster and alternative designs in the case of pandemics. Am. J. Bioeth. 13 23–24.
  • Wang, H., Yip, W., Zhang, L. and Hsiao, W. C. (2009). The impact of rural mutual health care on health status: Evaluation of a social experiment in rural China. Health Econ. 18 S65–S82.
  • Welch, B. L. (1937). On the z-test in randomized blocks and latin squares. Biometrika 29 21–52.
  • Woertman, W., de Hoop, E., Moerbeek, M., Zuidema, S. U., Gerritsen, D. L. and Teerenstra, S. (2013). Stepped wedge designs could reduce the required sample size in cluster randomized trials. J. Clin. Epidemiol. 66 752–758.