The Annals of Applied Statistics

Sequential advantage selection for optimal treatment regime

Ailin Fan, Wenbin Lu, and Rui Song

Full-text: Open access


Variable selection is gaining more attention because it plays an important role in deriving practical and reliable optimal treatment regimes for personalized medicine, especially when there are a large number of predictors. Most existing variable selection techniques focus on selecting variables that are important for prediction. With such methods, some variables that are poor in prediction but are critical for treatment decision making may be ignored. A qualitative interaction of a variable with treatment arises when the treatment effect changes direction as the value of the variable varies. Variables that have qualitative interactions with treatment are of clinical importance for treatment decision making. Gunter, Zhu and Murphy [Stat. Methodol. 8 (2011) 42–55] proposed the S-score method to characterize the magnitude of qualitative interaction of an individual variable with treatment. In this paper, we develop a sequential advantage selection method based on a modified S-score. Our method sequentially selects variables with a qualitative interaction and can be applied in multiple decision-point settings. To select the best candidate subset of variables for decision making, we also propose a BIC-type criterion that is based on the sequential advantage. The empirical performance of the proposed method is evaluated by simulation and an application to depression data from a clinical trial.

Article information

Ann. Appl. Stat., Volume 10, Number 1 (2016), 32-53.

Received: March 2015
First available in Project Euclid: 25 March 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Optimal treatment regime qualitative interaction sequential advantage variable selection


Fan, Ailin; Lu, Wenbin; Song, Rui. Sequential advantage selection for optimal treatment regime. Ann. Appl. Stat. 10 (2016), no. 1, 32--53. doi:10.1214/15-AOAS849.

Export citation


  • Chakraborty, B., Murphy, S. and Strecher, V. (2010). Inference for nonregular parameters in optimal dynamic treatment regimes. Stat. Methods Med. Res. 19 317–343.
  • Fava, M., Rush, A. J., Trivedi, M. H., Nierenberg, A. A., Thase, M. E., Sackeim, H. A., Quitkin, F. M., Wisniewski, S., Lavori, P. W., Rosenbaum, J. F. et al. (2003). Background and rationale for the sequenced treatment alternatives to relieve depression (STAR∗D) study. Psychiatric Clinics of North America 26 457–494.
  • Gail, M. and Simon, R. (1985). Testing for qualitative interactions between treatment effects and patient subsets. Biometrics 41 361–372.
  • Gunter, L., Zhu, J. and Murphy, S. A. (2011). Variable selection for qualitative interactions. Stat. Methodol. 8 42–55.
  • Lu, W., Zhang, H. H. and Zeng, D. (2013). Variable selection for optimal treatment decision. Stat. Methods Med. Res. 22 493–504.
  • Moodie, E. E. M., Richardson, T. S. and Stephens, D. A. (2007). Demystifying optimal dynamic treatment regimes. Biometrics 63 447–455.
  • Murphy, S. A. (2003). Optimal dynamic treatment regimes. J. R. Stat. Soc. Ser. B. Stat. Methodol. 65 331–366.
  • Murphy, S. A. (2005a). An experimental design for the development of adaptive treatment strategies. Stat. Med. 24 1455–1481.
  • Murphy, S. A. (2005b). A generalization error for Q-learning. J. Mach. Learn. Res. 6 1073–1097.
  • Murphy, S. A., van der Laan, M. J. and Robins, J. M. (2001). Marginal mean models for dynamic regimes. J. Amer. Statist. Assoc. 96 1410–1423.
  • Peto, R. (1982). Statistical aspects of cancel trials. In Treatment of Cancer (K. E. Halnan, ed.) 867–871. Chapman, London, UK.
  • Piantadosi, S. and Gail, M. H. (1993). A comparison of the power of two tests for qualitative interactions. Stat. Med. 12 1239–1248.
  • Qian, M. and Murphy, S. A. (2011). Performance guarantees for individualized treatment rules. Ann. Statist. 39 1180–1210.
  • Qian, M., Nahum-Shani, I. and Murphy, S. A. (2013). Dynamic treatment regimes. In Modern Clinical Trial Analysis 127–148. Springer, New York.
  • Robins, J. (1986). A new approach to causal inference in mortality studies with a sustained exposure period—Application to control of the healthy worker survivor effect. Math. Modelling 7 1393–1512.
  • Robins, J. M. (1997). Causal inference from complex longitudinal data. In Latent Variable Modeling and Applications to Causality (Los Angeles, CA, 1994). Lecture Notes in Statist. 120 69–117. Springer, New York.
  • Robins, J. M. (2004). Optimal structural nested models for optimal sequential decisions. In Proceedings of the Second Seattle Symposium in Biostatistics. Lecture Notes in Statist. 179 189–326. Springer, New York.
  • Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Ann. Statist. 6 34–58.
  • Rush, A. J., Fava, M., Wisniewski, S. R., Lavori, P. W., Trivedi, M. H., Sackeim, H. A., Thase, M. E., Nierenberg, A. A., Quitkin, F. M., Kashner, T. M. et al. (2004). Sequenced treatment alternatives to relieve depression (STAR∗D): Rationale and design. Controlled Clinical Trials 25 119–142.
  • Song, R., Wang, W., Zeng, D. and Kosorok, M. R. (2015). Penalized Q-learning for dynamic treatment regimens. Statist. Sinica 25 901–920.
  • Watkins, C. J. (1989). Learning from delayed rewards. Ph.D. thesis, Univ. Cambridge, England.
  • Watkins, C. J. and Dayan, P. (1992). Q-learning. Mach. Learn. 8 279–292.
  • Yan, X. (2004). Test for qualitative interaction in equivalence trials when the number of centres is large. Stat. Med. 23 711–722.
  • Zhang, B., Tsiatis, A. A., Laber, E. B. and Davidian, M. (2012a). A robust method for estimating optimal treatment regimes. Biometrics 68 1010–1018.
  • Zhang, B., Tsiatis, A. A., Davidian, M., Zhang, M. and Laber, E. (2012b). Estimating optimal treatment regimes from a classification perspective. Stat. 1 103–114.
  • Zhang, B., Tsiatis, A. A., Laber, E. B. and Davidian, M. (2013). Robust estimation of optimal dynamic treatment regimes for sequential treatment decisions. Biometrika 100 681–694.
  • Zhao, Y., Zeng, D., Socinski, M. A. and Kosorok, M. R. (2011). Reinforcement learning strategies for clinical trials in nonsmall cell lung cancer. Biometrics 67 1422–1433.
  • Zhao, Y., Zeng, D., Rush, A. J. and Kosorok, M. R. (2012). Estimating individualized treatment rules using outcome weighted learning. J. Amer. Statist. Assoc. 107 1106–1118.