Abstract
Classical Randomized Controlled Trials (RCTs), or A/B tests, are designed to draw causal inferences about a population of units, for example, individuals, plots of land or visits to a website. A key assumption underlying a standard RCT is the absence of interactions between units, or the stable unit treatment value assumption (Ann. Statist. 6 (1978) 34–58). Modern experimentation, however, is often conducted in settings characterized by complex interactions between units. Such interactions can invalidate the standard estimators and make classical experimental designs ineffective. Although the presence of interference forces us to make untestable assumptions on the nature of the interactions even under randomization, sophisticated experimental designs can ameliorate the dependence on such assumptions. In this manuscript, we review the recent and rapidly growing literature on novel experimental designs for these settings. One key feature common to many of these designs is the presence of multiple layers of randomization within the same experiment. We discuss a novel experimental design, called Multiple Randomization Designs or MRDs, that provides a general framework for such experiments. Through these complex designs, we can study questions about causal effects in the presence of interference that cannot be answered by classical RCTs.
Acknowledgments
We are grateful for discussions with Susan Athey, John Geweke, Matt Taddy, and Johan Ugander and for comments by participants in the following conferences: CODE@MIT, NABE (2019), ASSA (2020) and ICSDS (2022). This research was carried out as part of Imbens’ and Richardson’s consulting relationship with Amazon. Brian Burdick contributed while working for Amazon.
Citation
Patrick Bajari. Brian Burdick. Guido W. Imbens. Lorenzo Masoero. James McQueen. Thomas S. Richardson. Ido M. Rosen. "Experimental Design in Marketplaces." Statist. Sci. 38 (3) 458 - 476, August 2023. https://doi.org/10.1214/23-STS883
Information