The Annals of Statistics

Oracle inequalities for inverse problems

L. Cavalier, G. K. Golubev, D. Picard, and A. B. Tsybakov

Full-text: Open access


We consider a sequence space model of statistical linear inverse problems where we need to estimate a function $f$ from indirect noisy observations. Let a finite set $\Lambda$ of linear estimators be given. Our aim is to mimic the estimator in $\Lambda$ that has the smallest risk on the true $f$. Under general conditions, we show that this can be achieved by simple minimization of an unbiased risk estimator, provided the singular values of the operator of the inverse problem decrease as a power law. The main result is a nonasymptotic oracle inequality that is shown to be asymptotically exact. This inequality can also be used to obtain sharp minimax adaptive results. In particular, we apply it to show that minimax adaptation on ellipsoids in the multivariate anisotropic case is realized by minimization of unbiased risk estimator without any loss of efficiency with respect to optimal nonadaptive procedures.

Article information

Ann. Statist., Volume 30, Number 3 (2002), 843-874.

First available in Project Euclid: 6 August 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation 62G20: Asymptotic properties

Statistical inverse problems oracle inequalities adaptive curve estimation model selection exact minimax constants


Cavalier, L.; Golubev, G. K.; Picard, D.; Tsybakov, A. B. Oracle inequalities for inverse problems. Ann. Statist. 30 (2002), no. 3, 843--874. doi:10.1214/aos/1028674843.

Export citation


  • AKAIKE, H. (1973). Information theory and an extension of the maximum likelihood principle. In Proc. 2nd Internat. Sy mp. Inform. Theory, Budapest (B. N. Petrov and F. Csaki, eds.) 267-281. Akademiai Kiado, Budapest.
  • BIRGÉ, L. (2001). An alternative point of view on Lepski's method. In State of the Art in Probability and Statistics. Festschrift for W. R. van Zwet (M. de Gunst, C. Klaassen and A. van der Vaart, eds.) 113-133. IMS, Hay ward, CA.
  • BIRGÉ, L. and MASSART, P. (2001). Gaussian model selection. J. Eur. Math. Soc. (JEMS) 3 203- 268.
  • CAVALIER, L. and TSy BAKOV, A. B. (2002). Sharp adaptation for inverse problems with random noise. Probab. Theory Related Fields. To appear. Available at
  • DONOHO, D. L. (1995). Nonlinear solution of linear inverse problems by wavelet-vaguelette decomposition. Appl. Comput. Harmon. Anal. 2 101-126.
  • DONOHO, D. L. and JOHNSTONE, I. M. (1994). Ideal spatial adaptation via wavelet shrinkage. Biometrika 81 425-455.
  • DONOHO, D. L. and JOHNSTONE, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200-1224.
  • DONOHO, D. L. and JOHNSTONE, I. M. (1996). Neoclassical minimax problems, thresholding and adaptive function estimation. Bernoulli 2 39-62.
  • GOLDENSHLUGER, A. and PEREVERZEV, S. V. (2000). Adaptive estimation of linear functionals in Hilbert scales from indirect white noise observations. Probab. Theory Related Fields 118 169-186.
  • GOLDENSHLUGER, A. and TSy BAKOV, A. B. (2001). Adaptive prediction and estimation in linear regression with infinitely many parameters. Ann. Statist. 29 1601-1619.
  • GOLUBEV, G. K. (1987). Adaptive asy mptotically minimax estimates of smooth signals. Problems Inform. Transmission 23 57-67.
  • GOLUBEV, G. K. (1992). Nonparametric estimation of smooth probability densities in L2. Problems Inform. Transmission 28 44-54.
  • GOLUBEV, G. K. and KHASMINSKII, R. Z. (1999). A statistical approach to some inverse boundary problems for partial differential equations. Problems Inform. Transmission 35 136-149.
  • GOLUBEV, G. K. and KHASMINSKII, R. Z. (2001). A statistical approach to the Cauchy problem for the Laplace equation. In State of the Art in Probability and Statistics. Festschrift for W. R. van Zwet (M. de Gunst, C. Klaassen and A. van der Vaart, eds.) 419-433. IMS, Hay ward, CA.
  • GOLUBEV, G. K. and NUSSBAUM, M. (1992). Adaptive spline estimates in a nonparametric regression model. Theory Probab. Appl. 37 521-529.
  • HÄRDLE, W. and MARRON, J. S. (1985). Optimal bandwidth selection in nonparametric regression function estimation. Ann. Statist. 13 1465-1481.
  • JOHNSTONE, I. M. (1999). Wavelet shrinkage for correlated data and inverse problems: adaptivity results. Statist. Sinica 9 51-83.
  • JOHNSTONE, I. M. and SILVERMAN, B. W. (1990). Speed of estimation in positron emission tomography and related inverse problems. Ann. Statist. 18 251-280.
  • KERKy ACHARIAN, G. and PICARD, D. (2002). Minimax or maxisets? Bernoulli 8 219-253.
  • KNEIP, A. (1994). Ordered linear smoothers. Ann. Statist. 22 835-866.
  • KOO, J.-Y. (1993). Optimal rates of convergence for nonparametric statistical inverse problems. Ann. Statist. 21 590-599.
  • KOROSTELEV, A. P. and TSy BAKOV, A. B. (1993). Minimax Theory of Image Reconstruction. Lecture Notes in Statist. 82. Springer, New York.
  • LI, K.-C. (1986). Asy mptotic optimality of CL and generalized cross-validation in ridge regression with application to spline smoothing. Ann. Statist. 14 1101-1112.
  • LI, K.-C. (1987). Asy mptotic optimality of CP, CL, cross-validation and generalized crossvalidation: discrete index set. Ann. Statist. 15 958-976.
  • MAIR, B. and RUy MGAART, F. H. (1996). Statistical inverse estimation in Hilbert scales. SIAM J. Appl. Math. 56 1424-1444.
  • MALLOWS, C. L. (1973). Some comments on Cp. Technometrics 15 661-675.
  • NEMIROVSKI, A. (2000). Topics in nonparametric statistics. Ecole d'Été de Probabilités de St. Flour XXVIII. Lecture Notes in Math. 1738 85-277. Springer, New York.
  • PINSKER, M. S. (1980). Optimal filtering of square integrable signals in Gaussian white noise. Problems Inform. Transmission 16 120-133.
  • POLy AK, B. T. and TSy BAKOV, A. B. (1990). Asy mptotic optimality of the Cp-test for the orthogonal series estimation of regression. Theory Probab. Appl. 35 293-306.
  • POLy AK, B. T. and TSy BAKOV, A. B. (1992). A family of asy mptotic optimal methods for choosing the estimate order in orthogonal series regression. Theory Probab. Appl. 37 471-481.
  • STEIN, E. and WEISS, G. (1971). Introduction to Fourier Analy sis on Euclidean Spaces. Princeton Univ. Press.
  • WAHBA, G. (1977). Practical approximate solutions to linear operator equations when the data are noisy. SIAM J. Numer. Anal. 14 651-667.
  • WAHBA, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia.