Electronic Journal of Statistics

Adaptive spectral regularizations of high dimensional linear models

Yuri Golubev

Full-text: Open access


This paper focuses on recovering an unknown vector β from the noisy data Y=Xβ+σξ, where X is a known n×p-matrix, ξ is a standard white Gaussian noise, and σ is an unknown noise level. In order to estimate β, a spectral regularization method is used, and our goal is to choose its regularization parameter with the help of the data Y. In this paper, we deal solely with regularization methods based on the so-called ordered smoothers (see [13]) and extend the oracle inequality from [11] to the case, where the noise level is unknown.

Article information

Electron. J. Statist., Volume 5 (2011), 1588-1617.

First available in Project Euclid: 23 November 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62C10: Bayesian problems; characterization of Bayes procedures
Secondary: 62G05: Estimation

spectral regularization excess risk ordered smoother empirical risk minimization oracle inequality


Golubev, Yuri. Adaptive spectral regularizations of high dimensional linear models. Electron. J. Statist. 5 (2011), 1588--1617. doi:10.1214/11-EJS649. https://projecteuclid.org/euclid.ejs/1322057437

Export citation


  • [1] Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle., Proc. 2nd Intern. Symp. Inf. Theory, Petrov P.N. and Csaki F. eds. Budapest. 267-281.
  • [2] Bauer, F. and Hohage T. (2005). A Lepski-type stopping rule for regularized Newton methods, Inverse Problems, 21, 1975–1991.
  • [3] Bauer, F., Mathé, P. and Pereverzev, S. (2007) Local solutions to inverse problems in geodesy., Journal of Geodesy, 81, no. 1, pp. 39-51.
  • [4] Birgé, L. and Massart, P. (2007) Minimal penalties for Gaussian model selection, Probab. Theory Relat. Fields, 138, 33–73.
  • [5] Bissantz, N., Hohage, T., Munk A. and Ruymgaart F. (2007). Convergence rates of general regularization methods for statistical inverse problems and applications., SIAM J. Numer. Anal., 45, no. 6, 2610–2636.
  • [6] Cavalier, L. and Golubev, Yu. (2006). Risk hull method and regularization by projections of ill-posed inverse problems, Ann. of Stat., 34, pp. 1653–1677.
  • [7] Cavalier L., Golubev G., Picard, D., Tsybakov B. (2002). Oracle inequalities for inverse problems., Annals of Stat., 30 No 3, 843–874.
  • [8] Dey, A.K., Ruymgaart, F. H. and Mair, B. A. (1996). Cross-validation for parameter selection in inverse estimation problems., Scandinavian Journal of Statistics, vol. 23, pp. 609-620.
  • [9] Engl, H.W., Hanke M. and Neubauer, A. (1996)., Regularization of Inverse Problems. Kluwer Academic Publishers.
  • [10] Golubev, Yu. (2004). The principle of penalized empirical risk in severely ill-posed problems., Probab. Theory and Relat. Fields, 130, 18–38.
  • [11] Golubev, Yu. (2010). On universal oracle inequalities related to high dimensional linear models., Ann. of Statist., 38, Number 5, 2751–2780.
  • [12] Herman, G. T. (2009), Fundamentals of computerized tomography: Image reconstruction from projection. 2nd edition, Springer.
  • [13] Kneip, A. (1994). Ordered linear smoothers, Ann. Statist., 22, pp. 835–866.
  • [14] Landweber, L. (1951). An iteration, formula for Fredholm integral equations of the first kind, Amer. J. Math., 73, 615–624.
  • [15] Loubes, J.-M. and Ludeña, C. (2008). Adaptive complexity regularization for linear inverse problems, Electronic J. of Statist., 2, 661–677.
  • [16] Mair B. and Ruymgaart F.H. (1996). Statistical estimation in Hilbert scale., SIAM J. Appl. Math., 56, no. 5, 1424–1444.
  • [17] Mathé, P. (2006). The Lepskii principle revised, Inverse Problemes, 22, no. 3, L11–L15.
  • [18] Pinsker, M.S. (1980). Optimal filtration of square-integrable signals in Gaussian noise., Problems Inform. Transmission. 16 120–133.
  • [19] Sullivan, F.O. (1986). A statistical perspective on ill-posed inverse problems, Statist. Sci., 1, no. 4, 501–527.
  • [20] Tikhonov, A.N. and Arsenin, V.A. (1977)., Solution of Ill-posed Problems. Preface by translation editor Fritz John. Scripta Series in Mathematics. V. H. Winston & Sons, Washington, D.C.: John Wiley & Sons, New York.
  • [21] Van der Vaart, A. and Wellner, J. (1996)., Weak convergence and empirical processes. Springer-Verlag, New York.