Open Access
2011 High-dimensional covariance estimation by minimizing 1-penalized log-determinant divergence
Pradeep Ravikumar, Martin J. Wainwright, Garvesh Raskutti, Bin Yu
Electron. J. Statist. 5: 935-980 (2011). DOI: 10.1214/11-EJS631


Given i.i.d. observations of a random vector Xp, we study the problem of estimating both its covariance matrix Σ*, and its inverse covariance or concentration matrix Θ*=(Σ*)1. When X is multivariate Gaussian, the non-zero structure of Θ* is specified by the graph of an associated Gaussian Markov random field; and a popular estimator for such sparse Θ* is the 1-regularized Gaussian MLE. This estimator is sensible even for for non-Gaussian X, since it corresponds to minimizing an 1-penalized log-determinant Bregman divergence. We analyze its performance under high-dimensional scaling, in which the number of nodes in the graph p, the number of edges s, and the maximum node degree d, are allowed to grow as a function of the sample size n. In addition to the parameters (p,s,d), our analysis identifies other key quantities that control rates: (a) the -operator norm of the true covariance matrix Σ*; and (b) the operator norm of the sub-matrix Γ*SS, where S indexes the graph edges, and Γ*=(Θ*)1(Θ*)1; and (c) a mutual incoherence or irrepresentability measure on the matrix Γ* and (d) the rate of decay 1/f(n,δ) on the probabilities {|Σ̂nijΣ*ij|>δ}, where Σ̂n is the sample covariance based on n samples. Our first result establishes consistency of our estimate Θ̂ in the elementwise maximum-norm. This in turn allows us to derive convergence rates in Frobenius and spectral norms, with improvements upon existing results for graphs with maximum node degrees $d=o(\sqrt{s})$. In our second result, we show that with probability converging to one, the estimate Θ̂ correctly specifies the zero pattern of the concentration matrix Θ*. We illustrate our theoretical results via simulations for various graphs and problem parameters, showing good correspondences between the theoretical predictions and behavior in simulations.


Download Citation

Pradeep Ravikumar. Martin J. Wainwright. Garvesh Raskutti. Bin Yu. "High-dimensional covariance estimation by minimizing 1-penalized log-determinant divergence." Electron. J. Statist. 5 935 - 980, 2011.


Published: 2011
First available in Project Euclid: 15 September 2011

zbMATH: 1274.62190
MathSciNet: MR2836766
Digital Object Identifier: 10.1214/11-EJS631

Primary: 62F12
Secondary: 62F30

Keywords: Concentration , Covariance , Gaussian graphical models , ℓ_1 regularization , precision , Sparsity

Rights: Copyright © 2011 The Institute of Mathematical Statistics and the Bernoulli Society

Back to Top