Bernoulli

  • Bernoulli
  • Volume 21, Number 2 (2015), 1200-1230.

On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA

Florentina Bunea and Luo Xiao

Full-text: Open access

Abstract

This work provides a unified analysis of the properties of the sample covariance matrix $\Sigma_{n}$ over the class of $p\times p$ population covariance matrices $\Sigma$ of reduced effective rank $r_{e}(\Sigma)$. This class includes scaled factor models and covariance matrices with decaying spectrum. We consider $r_{e}(\Sigma)$ as a measure of matrix complexity, and obtain sharp minimax rates on the operator and Frobenius norm of $\Sigma_{n}-\Sigma$, as a function of $r_{e}(\Sigma)$ and $\|\Sigma\|_{2}$, the operator norm of $\Sigma$. With guidelines offered by the optimal rates, we define classes of matrices of reduced effective rank over which $\Sigma_{n}$ is an accurate estimator. Within the framework of these classes, we perform a detailed finite sample theoretical analysis of the merits and limitations of the empirical scree plot procedure routinely used in PCA. We show that identifying jumps in the empirical spectrum that consistently estimate jumps in the spectrum of $\Sigma$ is not necessarily informative for other goals, for instance for the selection of those sample eigenvalues and eigenvectors that are consistent estimates of their population counterparts. The scree plot method can still be used for selecting consistent eigenvalues, for appropriate threshold levels. We provide a threshold construction and also give a rule for checking the consistency of the corresponding sample eigenvectors. We specialize these results and analysis to population covariance matrices with polynomially decaying spectra, and extend it to covariance operators with polynomially decaying spectra. An application to fPCA illustrates how our results can be used in functional data analysis.

Article information

Source
Bernoulli, Volume 21, Number 2 (2015), 1200-1230.

Dates
First available in Project Euclid: 21 April 2015

Permanent link to this document
https://projecteuclid.org/euclid.bj/1429624975

Digital Object Identifier
doi:10.3150/14-BEJ602

Mathematical Reviews number (MathSciNet)
MR3338661

Zentralblatt MATH identifier
06445972

Keywords
covariance matrix eigenvalue eigenvector fPCA high dimensions minimax rate optimal rate of convergence PCA scree plot sparsity

Citation

Bunea, Florentina; Xiao, Luo. On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA. Bernoulli 21 (2015), no. 2, 1200--1230. doi:10.3150/14-BEJ602. https://projecteuclid.org/euclid.bj/1429624975


Export citation

References

  • [1] Anderson, T.W. (1963). Asymptotic theory for principal component analysis. Ann. Math. Statist. 34 122–148.
  • [2] Bai, J. and Ng, S. (2002). Determining the number of factors in approximate factor models. Econometrica 70 191–221.
  • [3] Baik, J. and Silverstein, J.W. (2006). Eigenvalues of large sample covariance matrices of spiked population models. J. Multivariate Anal. 97 1382–1408.
  • [4] Benko, M., Härdle, W. and Kneip, A. (2009). Common functional principal components. Ann. Statist. 37 1–34.
  • [5] Bickel, P.J. and Levina, E. (2008). Covariance regularization by thresholding. Ann. Statist. 36 2577–2604.
  • [6] Bickel, P.J. and Levina, E. (2008). Regularized estimation of large covariance matrices. Ann. Statist. 36 199–227.
  • [7] Bunea, F., She, Y. and Wegkamp, M.H. (2011). Optimal selection of reduced rank estimators of high-dimensional matrices. Ann. Statist. 39 1282–1309.
  • [8] Bunea, F. and Xiao, L. (2014). Supplement to “On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA.” DOI:10.3150/14-BEJ602SUPP.
  • [9] Cai, T. and Liu, W. (2011). Adaptive thresholding for sparse covariance matrix estimation. J. Amer. Statist. Assoc. 106 672–684.
  • [10] Cai, T.T., Zhang, C.-H. and Zhou, H.H. (2010). Optimal rates of convergence for covariance matrix estimation. Ann. Statist. 38 2118–2144.
  • [11] Chamberlain, G. and Rothschild, M. (1983). Arbitrage, factor structure, and mean-variance analysis on large asset markets. Econometrica 51 1281–1304.
  • [12] Dauxois, J., Pousse, A. and Romain, Y. (1982). Asymptotic theory for the principal component analysis of a vector random function: Some applications to statistical inference. J. Multivariate Anal. 12 136–154.
  • [13] Fan, J., Liao, Y. and Mincheva, M. (2013). Large covariance estimation by thresholding principal orthogonal complements. J. R. Stat. Soc. Ser. B Stat. Methodol. 75 603–680.
  • [14] Hall, P. and Hosseini-Nasab, M. (2006). On properties of functional principal components analysis. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 109–126.
  • [15] Hall, P., Müller, H.-G. and Wang, J.-L. (2006). Properties of principal component methods for functional and longitudinal data analysis. Ann. Statist. 34 1493–1517.
  • [16] Horn, R.A. and Johnson, C.R. (1985). Matrix Analysis. Cambridge: Cambridge Univ. Press.
  • [17] Johnstone, I.M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295–327.
  • [18] Juditsky, A.B. and Nemirovski, A.S. (2008). Large deviations of vector-values martingales in 2-smooth normed spaces. Preprint. Available at arXiv:0809.0813.
  • [19] Kneip, A. and Sarda, P. (2011). Factor models and variable selection in high-dimensional regression analysis. Ann. Statist. 39 2410–2447.
  • [20] Kneip, A. and Utikal, K.J. (2001). Inference for density families using functional principal component analysis. J. Amer. Statist. Assoc. 96 519–542.
  • [21] Lounici, K. (2013). High-dimensional covariance matrix estimation with missing observations. Bernoulli. To appear.
  • [22] Muirhead, R.J. (1982). Aspects of Multivariate Statistical Theory. Wiley Series in Probability and Mathematical Statistics. New York: Wiley.
  • [23] Nadler, B. (2008). Finite sample approximation results for principal component analysis: A matrix perturbation approach. Ann. Statist. 36 2791–2817.
  • [24] Tropp, J.A. (2012). User-friendly tail bounds for sums of random matrices. Found. Comput. Math. 12 389–434.
  • [25] Vershynin, R. (2012). Introduction to the non-asymptotic analysis of random matrices. In Compressed Sensing 210–268. Cambridge: Cambridge Univ. Press.
  • [26] Vu, V. and Lei, J. (2012. Minimax rates of estimation for sparse PCA in high dimensions. JMLR: Workshop and Conference Proceedings 22 1278–1286.
  • [27] Yao, F., Müller, H.-G. and Wang, J.-L. (2005). Functional data analysis for sparse longitudinal data. J. Amer. Statist. Assoc. 100 577–590.

Supplemental materials

  • Supplementary material: Supplement to “On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA”. We provide proofs of all the lemmas, propositions and theorems stated, but not proved, in the Appendix of the main paper.