The Annals of Statistics

Risk hull method and regularization by projections of ill-posed inverse problems

L. Cavalier and Yu. Golubev

Full-text: Open access


We study a standard method of regularization by projections of the linear inverse problem Y=Af+ε, where ε is a white Gaussian noise, and A is a known compact operator with singular values converging to zero with polynomial decay. The unknown function f is recovered by a projection method using the singular value decomposition of A. The bandwidth choice of this projection regularization is governed by a data-driven procedure which is based on the principle of risk hull minimization. We provide nonasymptotic upper bounds for the mean square risk of this method and we show, in particular, that in numerical simulations this approach may substantially improve the classical method of unbiased risk estimation.

Article information

Ann. Statist., Volume 34, Number 4 (2006), 1653-1677.

First available in Project Euclid: 3 November 2006

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation 62G20: Asymptotic properties

Inverse problem quadratic risk risk hull oracle inequality


Cavalier, L.; Golubev, Yu. Risk hull method and regularization by projections of ill-posed inverse problems. Ann. Statist. 34 (2006), no. 4, 1653--1677. doi:10.1214/009053606000000542.

Export citation


  • Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory (B. N. Petrov and F. Csáki, eds.) 267--281. Akadémiai Kiadó, Budapest.
  • Barron, A., Birgé, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301--413.
  • Bertero, M. and Boccacci, P. (1998). Introduction to Inverse Problems in Imaging. Institute of Physics Publishing, Bristol.
  • Birgé, L. and Massart, P. (2001). Gaussian model selection. J. Eur. Math. Soc. 3 203--268.
  • Cavalier, L., Golubev, G. K., Picard, D. and Tsybakov, A. B. (2002). Oracle inequalities for inverse problems. Ann. Statist. 30 843--874.
  • Cavalier, L., Golubev, Yu., Lepski, O. and Tsybakov, A. (2003). Block thresholding and sharp adaptive estimation in severely ill-posed inverse problems. Theory Probab. Appl. 48 426--446.
  • Cavalier, L. and Tsybakov, A. B. (2002). Sharp adaptation for inverse problems with random noise. Probab. Theory Related Fields 123 323--354.
  • Donoho, D. L. (1995). Nonlinear solutions of linear inverse problems by wavelet--vaguelette decomposition. Appl. Comput. Harmon. Anal. 2 101--126.
  • Efromovich, S. (1997). Robust and efficient recovery of a signal passed through a filter and then contaminated by non-Gaussian noise. IEEE Trans. Inform. Theory 43 1184--1191.
  • Engl, H. W., Hanke, M. and Neubauer, A. (1996). Regularization of Inverse Problems. Kluwer, Dordrecht.
  • Fan, J. (1991). On the optimal rates of convergence for nonparametric deconvolution problems. Ann. Statist. 19 1257--1272.
  • Golubev, Yu. (2004). The principle of penalized empirical risk in severely ill-posed problems. Probab. Theory Related Fields 130 18--38.
  • Golubev, Yu. and Levit, B. (2004). An oracle approach to adaptive estimation of linear functionals in a Gaussian model. Math. Methods Statist. 13 392--408.
  • Hackbusch, W. (1995). Integral Equations. Theory and Numerical Treatment. Birkhäuser, Basel.
  • Hida, T. (1980). Brownian Motion. Springer, New York.
  • Johnstone, I. M. (1999). Wavelet shrinkage for correlated data and inverse problems: Adaptivity results. Statist. Sinica 9 51--83.
  • Johnstone, I. M. and Silverman, B. W. (1990). Speed of estimation in positron emission tomography and related inverse problems. Ann. Statist. 18 251--280.
  • Lavrentiev, M. M. (1967). Some Improperly Posed Problems of Mathematical Physics. Springer, Berlin.
  • Mair, B. and Ruymgaart, F. H. (1996). Statistical inverse estimation in Hilbert scales. SIAM J. Appl. Math. 56 1424--1444.
  • Mallows, C. L. (1973). Some comments on $C_p$. Technometrics 15 661--675.
  • O'Sullivan, F. (1986). A statistical perspective on ill-posed inverse problems. Statist. Sci. 1 502--527.
  • Stein, C. M. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135--1151.
  • Tikhonov, A. N. and Arsenin, V. Y. (1977). Solutions of Ill-Posed Problems. Winston, Washington.