Open Access
February 2006 Classifier Technology and the Illusion of Progress
David J. Hand
Statist. Sci. 21(1): 1-14 (February 2006). DOI: 10.1214/088342306000000060


A great many tools have been developed for supervised classification, ranging from early methods such as linear discriminant analysis through to modern developments such as neural networks and support vector machines. A large number of comparative studies have been conducted in attempts to establish the relative superiority of these methods. This paper argues that these comparisons often fail to take into account important aspects of real problems, so that the apparent superiority of more sophisticated methods may be something of an illusion. In particular, simple methods typically yield performance almost as good as more sophisticated methods, to the extent that the difference in performance may be swamped by other sources of uncertainty that generally are not considered in the classical supervised classification paradigm.


Download Citation

David J. Hand. "Classifier Technology and the Illusion of Progress." Statist. Sci. 21 (1) 1 - 14, February 2006.


Published: February 2006
First available in Project Euclid: 6 June 2006

zbMATH: 05191849
MathSciNet: MR2275965
Digital Object Identifier: 10.1214/088342306000000060

Keywords: empirical comparisons , error rate , flat maximum effect , misclassification rate , population drift , principle of parsimony , problem uncertainty , selectivity bias , simplicity , Supervised classification

Rights: Copyright © 2006 Institute of Mathematical Statistics

Vol.21 • No. 1 • February 2006
Back to Top