Electronic Journal of Statistics

Bootstrapping Aalen-Johansen processes for competing risks: Handicaps, solutions, and limitations

Dennis Dobler and Markus Pauly

Full-text: Open access


Statistical inference in competing risks models is often based on the famous Aalen-Johansen estimator. Since the corresponding limit process lacks independent increments, it is typically applied together with Lin’s (1997) resampling technique involving standard normal multipliers. Recently, it has been seen that this approach can be interpreted as a wild bootstrap technique and that other multipliers, e.g. centered Poissons, may lead to better finite sample performances, see Beyersmann et al. (2013). Since the latter is closely related to Efron’s classical bootstrap, the question arises whether this or more general weighted bootstrap versions of Aalen-Johansen processes lead to valid results. Here we analyze their asymptotic behaviour and it turns out that such weighted bootstrap versions in general possess the wrong covariance structure in the limit. However, we demonstrate that the weighted bootstrap can nevertheless be applied for specific null hypotheses of interest and also discuss its limitations for statistical inference. To this end, we introduce different consistent weighted bootstrap tests for the null hypothesis of stochastically ordered cumulative incidence functions and compare their finite sample performance in a simulation study.

Article information

Electron. J. Statist., Volume 8, Number 2 (2014), 2779-2803.

First available in Project Euclid: 8 January 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G09: Resampling methods 62N03: Testing 62G10: Hypothesis testing
Secondary: 62M02: Markov processes: hypothesis testing

Aalen-Johansen estimator bootstrap competing risk counting processes cumulative incidence function left-truncation right-censoring weighted bootstrap


Dobler, Dennis; Pauly, Markus. Bootstrapping Aalen-Johansen processes for competing risks: Handicaps, solutions, and limitations. Electron. J. Statist. 8 (2014), no. 2, 2779--2803. doi:10.1214/14-EJS972. https://projecteuclid.org/euclid.ejs/1420726191

Export citation


  • [1] Aalen, O. O. and Johansen, S., An empirical transition matrix for non-homogeneous Markov chains based on censored observations., Scand. J. Statist., 5(3):141–150, 1978.
  • [2] Adler, R. J., An Introduction to Continuity, Extrema, and Related Topics for General Gaussian Processes. Institute of Mathematical Statistics Lecture Notes—Monograph Series, 12. Institute of Mathematical Statistics, Hayward, CA, 1990.
  • [3] Allignol, A., Schumacher, M., and Beyersmann, J., A note on variance estimation of the Aalen-Johansen estimator of the cumulative incidence function in competing risks, with a view towards left-truncated data., Biom. J., 52(1):126–137, 2010.
  • [4] Aly, E.-E. A. A., Kochar, S. C., and McKeague, I. W., Some tests for comparing cumulative incidence functions and cause-specific hazard rates., J. Amer. Statist. Assoc., 89:994–999, 1994.
  • [5] Andersen, P. K., Borgan, Ø., Gill, R. D., and Keiding, N., Statistical Models Based on Counting Processes. Springer, New York, 1993.
  • [6] Bajorunaite, R. and Klein, J. P., Two-sample tests of the equality of two cumulative incidence functions., Comput. Statist. Data Anal., 51 :4269–4281, 2007.
  • [7] Bajorunaite, R. and Klein, J. P., Comparison of failure probabilities in the presence of competing risks., J. Stat. Comput. Simul., 78:951–966, 2008.
  • [8] Beyersmann, J., Allignol, A., and Schumacher, M., Competing Risks and Multistate Models with R. Springer, New York, 2012.
  • [9] Beyersmann, J., Latouche, A., Buchholz, A., and Schumacher, M., Simulating competing risks data in survival analysis., Statistics in Medicine, 28(6):956–971, 2009.
  • [10] Beyersmann, J., Pauly, M., and Di Termini, S., Weak convergence of the wild bootstrap for the Aalen-Johansen estimator of the cumulative incidence function of a competing risk., Scand. J. Statist., 78:387–402, 2013.
  • [11] Billingsley, P., Convergence of Probability Measures. Wiley, New York, second edition, 1999.
  • [12] Brendel, M., Janssen, A., Mayer, C.-D., and Pauly, M., Weighted logrank permutation tests for randomly right censored life science data., Scand. J. Statist., 41(3):742–761, 2014.
  • [13] Cai, T., Tian, L., Uno, H., Solomon, S., and Wei, L., Calibrating parametric subject-specific risk estimation., Biometrika, 97:389–404, 2010.
  • [14] del Barrio, E., Janssen, A., and Matrán, C., Resampling schemes with low resampling intensity and their applications in testing hypotheses., J. Statist. Plann. Inference, 139(2):184–202, 2009.
  • [15] Efron, B., Bootstrap methods: Another look at the jackknife., Ann. Statist., 7(1):1–26, 1979.
  • [16] El Barmi, H., Kochar, S., and Tsimikas, J., Likelihood ratio tests for and against ordering of the cumulative incidence functions in multiple competing risks and discrete mark variable models., J. Statist. Plann. Inference, 136 :1588–1607, 2006.
  • [17] Gray, R. J., A class of $K$-sample tests for comparing the cumulative incidence of a competing risk., Ann. Statist., 16 :1141–1154, 1988.
  • [18] Janssen, A., Studentized permutation tests for non-i.i.d. hypotheses and the generalized Behrens-Fisher problem., Statist. Probab. Lett., 36(1):9–21, 1997.
  • [19] Janssen, A., Nonparametric symmetry tests for statistical functionals., Math. Methods Statist., 8(3):320–343, 1999.
  • [20] Janssen, A., Resampling Student’s $t$-type statistics., Ann. Inst. Statist. Math., 57(3):507–529, 2005.
  • [21] Janssen, A. and Pauls, T., How do bootstrap and permutation tests work?, Ann. Statist., 31(3):768–806, 2003.
  • [22] Johnson, N. L., Kotz, S., and Balakrishnan, N., Discrete Multivariate Distributions. Wiley Series in Probability and Statistics: Applied Probability and Statistics. A Wiley-Interscience Publication. John Wiley & Sons Inc., New York, 1997.
  • [23] Konietschke F. and Pauly, M., Bootstrapping and permuting paired $t$-test type statistics., Statistics and Computing, 24(3):283–296, 2014.
  • [24] Lin, D. Y., Non-parametric inference for cumulative incidence functions in competing risks studies., Statistics and Medicine, 16:901–910, 1997.
  • [25] Lin, D. Y., Wei, L. J., Yang, I., and Ying, Z., Semiparametric regression for the mean and rate functions of recurrent events., J. R. Stat. Soc. Ser. B Stat. Methodol., 62(4):711–730, 2000.
  • [26] Lin, D. Y., Wei, L. J., and Ying, Z., Checking the Cox model with cumulative sums of martingale-based residuals., Biometrika, 80(3):557–572, 1993.
  • [27] Mammen, E., When Does Bootstrap Work? Asymptotic Results and Simulations. Springer, New York, 1992.
  • [28] Martinussen, T. and Scheike, T. H., Dynamic Regression Models for Survival Data. Statistics for Biology and Health. Springer, New York, 2006.
  • [29] Mason, D. M. and Newton, M. A., A rank statistics approach to the consistency of a general bootstrap., Ann. Statist., 20(3) :1611–1624, 1992.
  • [30] Pauly, M., Eine Analyse bedingter Tests mit bedingten Zentralen Grenzwertsätzen für Resampling-Statistiken (in german)., Phd-thesis, University Düsseldorf, 2009.
  • [31] Pauly, M., Weighted resampling of martingale difference arrays with applications., Electronic Journal of Statistics, 5:41–52, 2011.
  • [32] Pauly, M., Brunner, E., and Konietschke, F., Asymptotic permutation tests in general factorial designs., Journal of the Royal Statistical Society – Series B, to appear, 2014.
  • [33] Pepe, M. S., Inference for events with dependent risks in multiple endpoint studies., Journal of the American Statistical Association, 86(415):770–778, 1991.
  • [34] Præstgaard, J. and Wellner, J. A., Exchangeably weighted bootstraps of the general empirical process., Ann. Probab., 21(4) :2053–2086, 1993.
  • [35] Putter, H. and van Zwet, W. R., Resampling: Consistency of substitution estimators., Ann. Statist., 24 :2297–2318, 1996.
  • [36] Sankaran, P. G., Unnikrishnan Nair, N., and Sreedevi, E. P., A quantile based test for comparing cumulative incidence functions of competing risks models., Statist. Probab. Lett., 80:886–891, 2010.
  • [37] Scheike, T. H. and Zhang, M.-J., Extensions and applications of the Cox-Aalen survival model., Biometrics, 59(4) :1036–1045, 2003.
  • [38] Shorack, G. R. and Wellner, J. A., Empirical Processes with Applications to Statistics. Society for Industrial and Applied Mathematics, Philadelphia, 2009.
  • [39] Spitoni, C., Verduijn, M., and Putter, H., Estimation and asymptotic theory for transition probabilities in markov renewal multi-state models., The International Journal of Biostatistics, 8, DOI:10.1515/1557-4679.1375, 2012.
  • [40] van der Vaart, A. W. and Wellner, J., Weak Convergence and Empirical Processes. Springer, New York et al., 1996.
  • [41] Wu, C.-F. J., Jackknife, bootstrap and other resampling methods in regression analysis., Ann. Statist., 14(4) :1261–1350, 1986. With discussion and a rejoinder by the author.