Open Access
November 2023 Methods for Integrating Trials and Non-experimental Data to Examine Treatment Effect Heterogeneity
Carly Lupton Brantner, Ting-Hsuan Chang, Trang Quynh Nguyen, Hwanhee Hong, Leon Di Stefano, Elizabeth A. Stuart
Author Affiliations +
Statist. Sci. 38(4): 640-654 (November 2023). DOI: 10.1214/23-STS890


Estimating treatment effects conditional on observed covariates can improve the ability to tailor treatments to particular individuals. Doing so effectively requires dealing with potential confounding, and also enough data to adequately estimate effect moderation. A recent influx of work has looked into estimating treatment effect heterogeneity using data from multiple randomized controlled trials and/or observational datasets. With many new methods available for assessing treatment effect heterogeneity using multiple studies, it is important to understand which methods are best used in which setting, how the methods compare to one another, and what needs to be done to continue progress in this field. This paper reviews these methods broken down by data setting: aggregate-level data, federated learning, and individual participant-level data. We define the conditional average treatment effect and discuss differences between parametric and nonparametric estimators, and we list key assumptions, both those that are required within a single study and those that are necessary for data combination. After describing existing approaches, we compare and contrast them and reveal open areas for future research. This review demonstrates that there are many possible approaches for estimating treatment effect heterogeneity through the combination of datasets, but that there is substantial work to be done to compare these methods through case studies and simulations, extend them to different settings, and refine them to account for various challenges present in real data.

Funding Statement

Research reported in this publication was partially funded through a Patient-Centered Outcomes Research Institute (PCORI) Award (ME-2020C3-21145; PI: Stuart) and by the National Institute of Mental Health (R01MH126856; PI: Stuart). Ms. Brantner also received financial support in the form of a training grant through the National Institutes of Health (T32AG000247). The statements in this work are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee, or of the National Institute of Mental Health.


The authors would like to thank the anonymous referees and the special issue Guest Editors for their constructive comments that improved the quality of this paper.

T.-H. Chang completed the work for this paper while employed as a Biostatistician at the Johns Hopkins Bloomberg School of Public Health.


Download Citation

Carly Lupton Brantner. Ting-Hsuan Chang. Trang Quynh Nguyen. Hwanhee Hong. Leon Di Stefano. Elizabeth A. Stuart. "Methods for Integrating Trials and Non-experimental Data to Examine Treatment Effect Heterogeneity." Statist. Sci. 38 (4) 640 - 654, November 2023.


Published: November 2023
First available in Project Euclid: 6 November 2023

Digital Object Identifier: 10.1214/23-STS890

Keywords: combining data , generalizability and reproducibility , treatment effect heterogeneity

Rights: Copyright © 2023 Institute of Mathematical Statistics

Vol.38 • No. 4 • November 2023
Back to Top