Electronic Journal of Statistics
- Electron. J. Statist.
- Volume 3 (2009), 651-677.
The false discovery rate for statistical pattern recognition
Clayton Scott, Gowtham Bellala, and Rebecca Willett
Abstract
The false discovery rate (FDR) and false nondiscovery rate (FNDR) have received considerable attention in the literature on multiple testing. These performance measures are also appropriate for classification, and in this work we develop generalization error analyses for FDR and FNDR when learning a classifier from labeled training data. Unlike more conventional classification performance measures, the empirical FDR and FNDR are not binomial random variables but rather a ratio of binomials, which introduces challenges not present in conventional formulations of the classification problem. We develop distribution-free uniform deviation bounds and apply these to obtain finite sample bounds and strong universal consistency. We also present a simulation study demonstrating the merits of variance-based bounds, which we also develop. In the context of multiple testing with FDR/FNDR, our framework may be viewed as a way to leverage training data to achieve distribution free, asymptotically optimal inference under the random effects model.
Article information
Source
Electron. J. Statist., Volume 3 (2009), 651-677.
Dates
First available in Project Euclid: 10 July 2009
Permanent link to this document
https://projecteuclid.org/euclid.ejs/1247231687
Digital Object Identifier
doi:10.1214/09-EJS363
Mathematical Reviews number (MathSciNet)
MR2521215
Zentralblatt MATH identifier
1326.62141
Subjects
Primary: 62H30: Classification and discrimination; cluster analysis [See also 68T10, 91C20]
Secondary: 68T05: Learning and adaptive systems [See also 68Q32, 91E40]
Keywords
Statistical learning theory generalization error false discovery rate
Citation
Scott, Clayton; Bellala, Gowtham; Willett, Rebecca. The false discovery rate for statistical pattern recognition. Electron. J. Statist. 3 (2009), 651--677. doi:10.1214/09-EJS363. https://projecteuclid.org/euclid.ejs/1247231687