Open Access
April 2012 General nonexact oracle inequalities for classes with a subexponential envelope
Guillaume Lecué, Shahar Mendelson
Ann. Statist. 40(2): 832-860 (April 2012). DOI: 10.1214/11-AOS965


We show that empirical risk minimization procedures and regularized empirical risk minimization procedures satisfy nonexact oracle inequalities in an unbounded framework, under the assumption that the class has a subexponential envelope function. The main novelty, in addition to the boundedness assumption free setup, is that those inequalities can yield fast rates even in situations in which exact oracle inequalities only hold with slower rates.

We apply these results to show that procedures based on $\ell_{1}$ and nuclear norms regularization functions satisfy oracle inequalities with a residual term that decreases like $1/n$ for every $L_{q}$-loss functions ($q\geq2$), while only assuming that the tail behavior of the input and output variables are well behaved. In particular, no RIP type of assumption or “incoherence condition” are needed to obtain fast residual terms in those setups. We also apply these results to the problems of convex aggregation and model selection.


Download Citation

Guillaume Lecué. Shahar Mendelson. "General nonexact oracle inequalities for classes with a subexponential envelope." Ann. Statist. 40 (2) 832 - 860, April 2012.


Published: April 2012
First available in Project Euclid: 1 June 2012

zbMATH: 1274.62247
MathSciNet: MR2933668
Digital Object Identifier: 10.1214/11-AOS965

Primary: 62G05
Secondary: 62H30 , 68T10

Keywords: Aggregation , ‎classification‎ , fast rates of convergence , High-dimensional data , Model selection , Oracle inequalities , regularization , Statistical learning

Rights: Copyright © 2012 Institute of Mathematical Statistics

Vol.40 • No. 2 • April 2012
Back to Top