Estimating linear and quadratic forms via indirect observations

Abstract

In this paper, we further develop the approach, originating in Juditsky and Nemirovski (Ann. Statist. 37 (2009) 2278–2300), to “computation-friendly” statistical estimation via Convex Programming.Our focus is on estimating a linear or quadratic form of an unknown “signal,” known to belong to a given convex compact set, via noisy indirect observations of the signal. Classical theoretical results on the subject deal with precisely stated statistical models and aim at designing statistical inferences and quantifying their performance in a closed analytic form. In contrast to this traditional (highly instructive) descriptive framework, the approach we promote here can be qualified as operational – the estimation routines and their risks are not available “in a closed form,” but are yielded by an efficient computation. All we know in advance is that under favorable circumstances the risk of the resulting estimate, whether high or low, is provably near-optimal under the circumstances. As a compensation for the lack of “explanatory power,” this approach is applicable to a much wider family of observation schemes than those where “closed form descriptive analysis” is possible.

We discuss applications of this approach to classical problems of estimating linear forms of parameters of sub-Gaussian distribution and quadratic forms of parameters of Gaussian and discrete distributions. The performance of the constructed estimates is illustrated by computation experiments in which we compare the risks of the constructed estimates with (numerical) lower bounds for corresponding minimax risks for randomly sampled estimation problems.