Open Access
2014 Aggregation of affine estimators
Dong Dai, Philippe Rigollet, Lucy Xia, Tong Zhang
Electron. J. Statist. 8(1): 302-327 (2014). DOI: 10.1214/14-EJS886

Abstract

We consider the problem of aggregating a general collection of affine estimators for fixed design regression. Relevant examples include some commonly used statistical estimators such as least squares, ridge and robust least squares estimators. Dalalyan and Salmon [DS12] have established that, for this problem, exponentially weighted (EW) model selection aggregation leads to sharp oracle inequalities in expectation, but similar bounds in deviation were not previously known. While results [DRZ12] indicate that the same aggregation scheme may not satisfy sharp oracle inequalities with high probability, we prove that a weaker notion of oracle inequality for EW that holds with high probability. Moreover, using a generalization of the newly introduced $Q$-aggregation scheme we also prove sharp oracle inequalities that hold with high probability. Finally, we apply our results to universal aggregation and show that our proposed estimator leads simultaneously to all the best known bounds for aggregation, including $\ell_{q}$-aggregation, $q\in(0,1)$, with high probability.

Citation

Download Citation

Dong Dai. Philippe Rigollet. Lucy Xia. Tong Zhang. "Aggregation of affine estimators." Electron. J. Statist. 8 (1) 302 - 327, 2014. https://doi.org/10.1214/14-EJS886

Information

Published: 2014
First available in Project Euclid: 10 April 2014

zbMATH: 1348.62132
MathSciNet: MR3192554
Digital Object Identifier: 10.1214/14-EJS886

Subjects:
Primary: 62G08
Secondary: 62C20 , 62G05 , 62G20

Keywords: affine estimators , Aggregation , Gaussian mean , Maurey’s argument , Oracle inequalities

Rights: Copyright © 2014 The Institute of Mathematical Statistics and the Bernoulli Society

Vol.8 • No. 1 • 2014
Back to Top