Open Access
November 2023 Replicability Across Multiple Studies
Marina Bogomolov, Ruth Heller
Author Affiliations +
Statist. Sci. 38(4): 602-620 (November 2023). DOI: 10.1214/23-STS892


Meta-analysis is routinely performed in many scientific disciplines. This analysis is attractive since discoveries are possible even when all the individual studies are underpowered. However, the meta-analytic discoveries may be entirely driven by signal in a single study, and thus nonreplicable. Although the great majority of meta-analyses carried out to date do not infer on the replicability of their findings, it is possible to do so. We provide a selective overview of analyses that can be carried out towards establishing replicability of the scientific findings. We describe methods for the setting where a single outcome is examined in multiple studies (as is common in systematic reviews of medical interventions), as well as for the setting where multiple studies each examine multiple features (as in genomics applications). We also discuss some of the current shortcomings and future directions.

Funding Statement

The first author was supported in part by Israeli Science Foundation grant no. 1726/19. The second author was supported in part by Israeli Science Foundation grant no. 2180/20.


The authors would like to thank the Guest Editors of the special issue of Statistical Science on Reproducibility and Replicability and the referee for their excellent suggestions. The authors are also grateful to Yoav Benjamini, Saharon Rosset, and Daniel Yekutieli for useful discussions that helped shape the paper.


Download Citation

Marina Bogomolov. Ruth Heller. "Replicability Across Multiple Studies." Statist. Sci. 38 (4) 602 - 620, November 2023.


Published: November 2023
First available in Project Euclid: 6 November 2023

Digital Object Identifier: 10.1214/23-STS892

Keywords: composite null , False discovery rate , Meta-analysis , multiple hypothesis testing , replicability analysis

Rights: Copyright © 2023 Institute of Mathematical Statistics

Vol.38 • No. 4 • November 2023
Back to Top