Open Access
August 2004 Complexity regularization via localized random penalties
Gábor Lugosi, Marten Wegkamp
Ann. Statist. 32(4): 1679-1697 (August 2004). DOI: 10.1214/009053604000000463


In this article, model selection via penalized empirical loss minimization in nonparametric classification problems is studied. Data-dependent penalties are constructed, which are based on estimates of the complexity of a small subclass of each model class, containing only those functions with small empirical loss. The penalties are novel since those considered in the literature are typically based on the entire model class. Oracle inequalities using these penalties are established, and the advantage of the new penalties over those based on the complexity of the whole model class is demonstrated.


Download Citation

Gábor Lugosi. Marten Wegkamp. "Complexity regularization via localized random penalties." Ann. Statist. 32 (4) 1679 - 1697, August 2004.


Published: August 2004
First available in Project Euclid: 4 August 2004

zbMATH: 1045.62060
MathSciNet: MR2089138
Digital Object Identifier: 10.1214/009053604000000463

Primary: 62G99 , 62H30
Secondary: 60E15

Keywords: ‎classification‎ , Complexity regularization , Concentration inequalities , Oracle inequalities , Rademacher averages , random penalties , shatter coefficients

Rights: Copyright © 2004 Institute of Mathematical Statistics

Vol.32 • No. 4 • August 2004
Back to Top