Electronic Journal of Statistics

On higher order isotropy conditions and lower bounds for sparse quadratic forms

Sara van de Geer and Alan Muro

Full-text: Open access

Abstract

This study aims at contributing to lower bounds for empirical compatibility constants or empirical restricted eigenvalues. This is of importance in compressed sensing and theory for $\ell_{1}$-regularized estimators. Let $X$ be an $n\times p$ data matrix with rows being independent copies of a $p$-dimensional random variable. Let $\hat{\Sigma}:=X^{T}X/n$ be the inner product matrix. We show that the quadratic forms $u^{T}\hat{\Sigma}u$ are lower bounded by a value converging to one, uniformly over the set of vectors $u$ with $u^{T}\Sigma_{0}u$ equal to one and $\ell_{1}$-norm at most $M$. Here $\Sigma_{0}:=\mathbb{E} \hat{\Sigma}$ is the theoretical inner product matrix, which we assume to exist. The constant $M$ is required to be of small order $\sqrt{n/\log p}$. We assume moreover $m$-th order isotropy for some $m>2$ and sub-exponential tails or moments up to order $\log p$ for the entries in $X$. As a consequence, we obtain convergence of the empirical compatibility constant to its theoretical counterpart, and similarly for the empirical restricted eigenvalue. If the data matrix $X$ is first normalized so that its columns all have equal length we obtain lower bounds assuming only isotropy and no further moment conditions on its entries. The isotropy condition is shown to hold for certain martingale situations.

Article information

Source
Electron. J. Statist., Volume 8, Number 2 (2014), 3031-3061.

Dates
First available in Project Euclid: 15 January 2015

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1421330629

Digital Object Identifier
doi:10.1214/15-EJS983

Mathematical Reviews number (MathSciNet)
MR3301300

Zentralblatt MATH identifier
1308.62148

Subjects
Primary: 62J07: Ridge regression; shrinkage estimators

Keywords
Compatibility isotropy quadratic forms restricted eigenvalue sparsity transfer principle

Citation

van de Geer, Sara; Muro, Alan. On higher order isotropy conditions and lower bounds for sparse quadratic forms. Electron. J. Statist. 8 (2014), no. 2, 3031--3061. doi:10.1214/15-EJS983. https://projecteuclid.org/euclid.ejs/1421330629


Export citation

References

  • [1] Adamczak, R., Litvak, A. E., Pajor, A., and Tomczak-Jaegermann, N., Quantitative estimates of the convergence of the empirical covariance matrix in log-concave ensembles., J. Amer. Math. Soc, 23(2):535–561, 2010.
  • [2] Bickel, P., Ritov, Y., and Tsybakov, A., Simultaneous analysis of Lasso and Dantzig selector., Annals of Statistics, 37 :1705–1732, 2009.
  • [3] Bousquet, O., A Bennet concentration inequality and its application to suprema of empirical processes., Comptes Rendus de l’Académie des Sciences, Paris, 334:495–550, 2002.
  • [4] Bühlmann, P. and van de Geer, S., Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer, 2011.
  • [5] Chen, S. S., Donoho, D. L., and Saunders, M. A., Atomic decomposition by basis pursuit., SIAM Journal on Scientific Computing, 20(1):33–61, 1998.
  • [6] Gribonval, R. and Nielsen, M., Highly sparse representations from dictionaries are unique and independent of the sparseness measure., Applied and Computational Harmonic Analysis, 22(3):335–355, 2007.
  • [7] Koltchinskii, V. and Mendelson, S., Bounding the smallest singular value of a random matrix without concentration., Arxiv preprint arXiv: 1312.3580, 2013.
  • [8] Lecué, G. and Mendelson, S., Necessary moment conditions for exact reconstruction via basis pursuit., Arxiv preprint arXiv: 404.3116, 2014.
  • [9] Lecué, G. and Mendelson, S., Compressed sensing under weak moment assumptions., Arxiv preprint arXiv: 1401.2188, 2014.
  • [10] Ledoux, M. and Talagrand, M., Probability in Banach Spaces: Isoperimetry and Processes. Springer Verlag, New York, 1991.
  • [11] Loh, P.-L. and Wainwright, M. J., High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity., Annals of Statistics, 40 :1637–1664, 2012.
  • [12] Mendelson, S., Pajor, A., and Tomczak-Jaegermann, N., Uniform uncertainty principle for Bernoulli and subgaussian ensembles., Constructive Approximation, 28(3):277–289, 2008.
  • [13] Oliveira, R. I., The lower tail of random quadratic forms, with applications to ordinary least squares and restricted eigenvalue properties., Arxiv preprint arXiv: 1312.2903, 2013.
  • [14] Raskutti, G., Wainwright, M. J., and Yu, B., Restricted eigenvalue properties for correlated Gaussian designs., Journal of Machine Learning Research, 11 :2241–2259, 2010.
  • [15] Rudelson, M. and Zhou, S., Reconstruction from anisotropic random measurements., Journal of Machine Learning Research – Proceedings Track 23, pages 10.1–10.24, 2012.
  • [16] Srivastava, N. and Vershynin, R., Covariance estimation for distributions with $2+\varepsilon$ moments., The Annals of Probability, 41(5) :3081–3111, 2013.
  • [17] van de Geer, S., The deterministic Lasso., The JSM Proceedings, 2007.
  • [18] van de Geer, S., On the uniform convergence of empirical norms and inner products, with application to causal inference., Electronic Journal of Statistics, 8:543–574, 2014.
  • [19] van de Geer, S. and Bühlmann, P., On the conditions used to prove oracle results for the Lasso., Electronic Journal of Statistics, pages 1360–1392, 2009.
  • [20] van der Vaart, A. W. and Wellner, J. A., Weak Convergence and Empirical Processes. Springer Series in Statistics. Springer-Verlag, New York, 1996. ISBN 0-387-94640-3.