Electronic Journal of Statistics

Sharp minimax tests for large covariance matrices and adaptation

Cristina Butucea and Rania Zgheib

Full-text: Open access

Abstract

We consider the detection problem of correlations in a $p$-dimensional Gaussian vector, when we observe $n$ independent, identically distributed random vectors, for $n$ and $p$ large. We assume that the covariance matrix varies in some ellipsoid with parameter $\alpha >1/2$ and total energy bounded by $L>0$.

We propose a test procedure based on a U-statistic of order 2 which is weighted in an optimal way. The weights are the solution of an optimization problem, they are constant on each diagonal and non-null only for the $T$ first diagonals, where $T=o(p)$. We show that this test statistic is asymptotically Gaussian distributed under the null hypothesis and also under the alternative hypothesis for matrices close to the detection boundary. We prove upper bounds for the total error probability of our test procedure, for $\alpha>1/2$ and under the assumption $T=o(p)$ which implies that $n=o(p^{2\alpha})$. We illustrate via a numerical study the behavior of our test procedure.

Moreover, we prove lower bounds for the maximal type II error and the total error probabilities. Thus we obtain the asymptotic and the sharp asymptotically minimax separation rate $\widetilde{\varphi}=(C(\alpha,L)n^{2}p)^{-\alpha/(4\alpha +1)}$, for $\alpha>3/2$ and for $\alpha >1$ together with the additional assumption $p=o(n^{4\alpha -1})$, respectively. We deduce rate asymptotic minimax results for testing the inverse of the covariance matrix.

We construct an adaptive test procedure with respect to the parameter $\alpha$ and show that it attains the rate $\widetilde{\psi}=(n^{2}p/\ln\ln(n\sqrt{p}))^{-\alpha/(4\alpha +1)}$.

Article information

Source
Electron. J. Statist., Volume 10, Number 2 (2016), 1927-1972.

Dates
Received: November 2015
First available in Project Euclid: 18 July 2016

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1468849967

Digital Object Identifier
doi:10.1214/16-EJS1143

Mathematical Reviews number (MathSciNet)
MR3522665

Zentralblatt MATH identifier
1346.62086

Subjects
Primary: 62G10: Hypothesis testing 62H15: Hypothesis testing
Secondary: 62G20: Asymptotic properties

Keywords
adaptive test covariance matrix goodness-of-fit tests high-dimensional data minimax separation rate sharp asymptotic rate U-statistic

Citation

Butucea, Cristina; Zgheib, Rania. Sharp minimax tests for large covariance matrices and adaptation. Electron. J. Statist. 10 (2016), no. 2, 1927--1972. doi:10.1214/16-EJS1143. https://projecteuclid.org/euclid.ejs/1468849967


Export citation

References

  • [1] Bai, Z., Jiang, D., Yao, J.-F. and Zheng, S. (2009). Corrections to LRT on large-dimensional covariance matrix by RMT., Ann. Statist. 37 3822–3840.
  • [2] Berthet, Q. and Rigollet, P. (2013). Optimal detection of sparse principal components in high dimension., Ann. Statist. 41 1780–1815.
  • [3] Bickel, P. J. and Levina, E. (2008a). Regularized estimation of large covariance matrices., Ann. Statist. 36 199–227.
  • [4] Bickel, P. J. and Levina, E. (2008b). Covariance regularization by thresholding., Ann. Statist. 36 2577–2604.
  • [5] Butucea, C. and Gayraud, G. (2013). Sharp detection of smooth signals in a high-dimensional sparse matrix with indirect observations., ArXiv e-prints.
  • [6] Butucea, C., Matias, C. and Pouet, C. (2009). Adaptive goodness-of-fit testing from indirect observations., Ann. Inst. Henri Poincaré Probab. Stat. 45 352–372.
  • [7] Butucea, C. and Zgheib, R. (2016). Sharp minimax tests for large Toeplitz covariance matrices with repeated observations., J. Multivariate Anal. 146 164–176.
  • [8] Cai, T. T. and Jiang, T. (2011). Limiting laws of coherence of random matrices with applications to testing covariance structure and construction of compressed sensing matrices., Ann. Statist. 39 1496–1525.
  • [9] Cai, T. T. and Ma, Z. (2013). Optimal hypothesis testing for high dimensional covariance matrices., Bernoulli 19 2359–2388.
  • [10] Cai, T. T. and Yuan, M. (2012). Adaptive covariance matrix estimation through block thresholding., Ann. Statist. 40 2014–2042.
  • [11] Cai, T. T., Zhang, C. H. and Zhou, H. H. (2010). Optimal rates of convergence for covariance matrix estimation., Ann. Statist. 38 2118–2144.
  • [12] Cai, T. T. and Zhou, H. H. (2012). Minimax estimation of large covariance matrices under $\ell_1$-norm., Statist. Sinica 22 1319–1349.
  • [13] Chen, S. X., Zhang, L. X. and Zhong, P. S. (2010). Tests for high-dimensional covariance matrices., J. Amer. Statist. Assoc. 105 810–819.
  • [14] Ermakov, M. S. (1990). A minimax test for hypotheses on a spectral density., Zap. Nauchn. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 184 115–125, 322.
  • [15] Golubev, G. K., Nussbaum, M. and Zhou, H. H. (2010). Asymptotic equivalence of spectral density estimation and Gaussian white noise., Ann. Statist. 38 181–214.
  • [16] Hall, P. (1984). Central limit theorem for integrated square error of multivariate nonparametric density estimators., J. Multivariate Anal. 14 1–16.
  • [17] Ingster, Y. I. (1993). Asymptotically minimax hypothesis testing for nonparametric alternatives. I., Math. Methods Statist. 2 85–114.
  • [18] Ingster, Y. I. and Suslina, I. A. (2003)., Nonparametric goodness-of-fit testing under Gaussian models. Lecture Notes in Statistics 169. Springer-Verlag, New York.
  • [19] Jiang, T. (2004). The asymptotic distributions of the largest entries of sample correlation matrices., Ann. Appl. Probab. 14 865–880.
  • [20] Ledoit, O. and Wolf, M. (2002). Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size., Ann. Statist. 30 1081–1102.
  • [21] Nagao, H. (1973). On some test criteria for covariance matrix., Ann. Statist. 1 700–709.
  • [22] Xiao, H. and Wu, W. B. (2013). Asymptotic theory for maximum deviations of sample covariance matrix estimates., Stochastic Process. Appl. 123 2899–2920.