Open Access
August 2012 Sharp oracle inequalities for aggregation of affine estimators
Arnak S. Dalalyan, Joseph Salmon
Ann. Statist. 40(4): 2327-2355 (August 2012). DOI: 10.1214/12-AOS1038


We consider the problem of combining a (possibly uncountably infinite) set of affine estimators in nonparametric regression model with heteroscedastic Gaussian noise. Focusing on the exponentially weighted aggregate, we prove a PAC-Bayesian type inequality that leads to sharp oracle inequalities in discrete but also in continuous settings. The framework is general enough to cover the combinations of various procedures such as least square regression, kernel ridge regression, shrinking estimators and many other estimators used in the literature on statistical inverse problems. As a consequence, we show that the proposed aggregate provides an adaptive estimator in the exact minimax sense without discretizing the range of tuning parameters or splitting the set of observations. We also illustrate numerically the good performance achieved by the exponentially weighted aggregate.


Download Citation

Arnak S. Dalalyan. Joseph Salmon. "Sharp oracle inequalities for aggregation of affine estimators." Ann. Statist. 40 (4) 2327 - 2355, August 2012.


Published: August 2012
First available in Project Euclid: 23 January 2013

zbMATH: 1257.62038
MathSciNet: MR3059085
Digital Object Identifier: 10.1214/12-AOS1038

Primary: 62G08
Secondary: 62C20 , 62G05 , 62G20

Keywords: Aggregation , exponentially weighted aggregation , minimax risk , Model selection , Oracle inequalities , regression

Rights: Copyright © 2012 Institute of Mathematical Statistics

Vol.40 • No. 4 • August 2012
Back to Top