May 2023 Confidence and Discoveries with E-values
Vladimir Vovk, Ruodu Wang
Author Affiliations +
Statist. Sci. 38(2): 329-354 (May 2023). DOI: 10.1214/22-STS874


We discuss systematically two versions of confidence regions: those based on p-values and those based on e-values, a recent alternative to p-values. Both versions can be applied to multiple hypothesis testing, and in this paper we are interested in procedures that control the number of false discoveries under arbitrary dependence between the base p- or e-values. We introduce a procedure that is based on e-values and show that it is efficient both computationally and statistically using simulated and real-world data sets. Comparison with the corresponding standard procedure based on p-values is not straightforward, but there are indications that the new one performs significantly better in some situations.

Funding Statement

V. Vovk’s research has been partially supported by Amazon, Astra Zeneca and Stena Line. R. Wang is supported by the Natural Sciences and Engineering Research Council of Canada (RGPIN-2018-03823, RGPAS-2018-522590).


We are grateful to Peter Westfall for his advice about the literature on Bayesian two-sample t-tests. We thank Glenn Shafer, Aaditya Ramdas and participants in the course “Game-theoretic statistics” (January–April 2021) for helpful comments. The presentation was greatly improved as result of the comments by two referees, an Associate Editor and the Editor (Sonia Petrone). For most of our simulation and empirical studies in Sections 7–8, we used Python. We also used the R package hommel (Goeman, Meijer and Krebs, 2019) and a data set available in the R package qvalue (Storey et al., 2019).


Download Citation

Vladimir Vovk. Ruodu Wang. "Confidence and Discoveries with E-values." Statist. Sci. 38 (2) 329 - 354, May 2023.


Published: May 2023
First available in Project Euclid: 6 February 2023

MathSciNet: MR4596762
zbMATH: 07708435
Digital Object Identifier: 10.1214/22-STS874

Keywords: Bayes factor , discovery matrix , discovery vector , Hypothesis testing , multiple hypothesis testing

Rights: Copyright © 2023 Institute of Mathematical Statistics


This article is only available to subscribers.
It is not available for individual sale.

Vol.38 • No. 2 • May 2023
Back to Top