The Annals of Probability

Necessary and sufficient conditions for the asymptotic distributions of coherence of ultra-high dimensional random matrices

Abstract

Let $\mathbf{x} _{1},\ldots,\mathbf{x}_{n}$ be a random sample from a $p$-dimensional population distribution, where $p=p_{n}\to\infty$ and $\log p=o(n^{\beta})$ for some $0<\beta\leq1$, and let $L_{n}$ be the coherence of the sample correlation matrix. In this paper it is proved that $\sqrt{n/\log p}L_{n}\to2$ in probability if and only if $Ee^{t_{0}|x_{11}|^{\alpha}}<\infty$ for some $t_{0}>0$, where $\alpha$ satisfies $\beta=\alpha/(4-\alpha)$. Asymptotic distributions of $L_{n}$ are also proved under the same sufficient condition. Similar results remain valid for $m$-coherence when the variables of the population are $m$ dependent. The proofs are based on self-normalized moderate deviations, the Stein–Chen method and a newly developed randomized concentration inequality.

Article information

Source
Ann. Probab., Volume 42, Number 2 (2014), 623-648.

Dates
First available in Project Euclid: 24 February 2014

https://projecteuclid.org/euclid.aop/1393251298

Digital Object Identifier
doi:10.1214/13-AOP837

Mathematical Reviews number (MathSciNet)
MR3178469

Zentralblatt MATH identifier
1354.60020

Citation

Shao, Qi-Man; Zhou, Wen-Xin. Necessary and sufficient conditions for the asymptotic distributions of coherence of ultra-high dimensional random matrices. Ann. Probab. 42 (2014), no. 2, 623--648. doi:10.1214/13-AOP837. https://projecteuclid.org/euclid.aop/1393251298

References

• Arratia, Goldstein and Gordon (1989) Arratia, R., Goldstein, L. and Gordon, L. (1989). Two moments suffice for Poisson approximations: The Chen–Stein method. Ann. Probab. 17 9–25.
• Cai and Jiang (2011) Cai, T. T. and Jiang, T. (2011). Limiting laws of coherence of random matrices with applications to testing covariance structure and construction of compressed sensing matrices. Ann. Statist. 39 1496–1525.
• Cai and Jiang (2012) Cai, T. T. and Jiang, T. (2012). Phase transition in limiting distributions of coherence of high-dimensional random matrices. J. Multivariate Anal. 107 24–39.
• Cai, Wang and Xu (2010) Cai, T. T., Wang, L. and Xu, G. (2010). Stable recovery of sparse signals and an oracle inequality. IEEE Trans. Inform. Theory 56 3516–3522.
• Donoho, Elad and Temlyakov (2006) Donoho, D. L., Elad, M. and Temlyakov, V. N. (2006). Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inform. Theory 52 6–18.
• Donoho and Huo (2001) Donoho, D. L. and Huo, X. (2001). Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inform. Theory 47 2845–2862.
• Fuchs (2004) Fuchs, J.-J. (2004). On sparse representations in arbitrary redundant bases. IEEE Trans. Inform. Theory 50 1341–1344.
• Jiang (2004) Jiang, T. (2004). The asymptotic distributions of the largest entries of sample correlation matrices. Ann. Appl. Probab. 14 865–880.
• Jing, Shao and Wang (2003) Jing, B.-Y., Shao, Q.-M. and Wang, Q. (2003). Self-normalized Cramér-type large deviations for independent random variables. Ann. Probab. 31 2167–2215.
• Li, Liu and Rosalsky (2010) Li, D., Liu, W.-D. and Rosalsky, A. (2010). Necessary and sufficient conditions for the asymptotic distribution of the largest entry of a sample correlation matrix. Probab. Theory Related Fields 148 5–35.
• Li, Qi and Rosalsky (2012) Li, D., Qi, Y. and Rosalsky, A. (2012). On Jiang’s asymptotic distribution of the largest entry of a sample correlation matrix. J. Multivariate Anal. 111 256–270.
• Li and Rosalsky (2006) Li, D. and Rosalsky, A. (2006). Some strong limit theorems for the largest entries of sample correlation matrices. Ann. Appl. Probab. 16 423–447.
• Linnik (1961) Linnik, J. V. (1961). On the probability of large deviations for the sums of independent variables. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. II 289–306. Univ. California Press, Berkeley, CA.
• Liu, Lin and Shao (2008) Liu, W.-D., Lin, Z. and Shao, Q.-M. (2008). The asymptotic distribution and Berry–Esseen bound of a new test for independence in high dimension with an application to stochastic optimization. Ann. Appl. Probab. 18 2337–2366.
• Shao (1997) Shao, Q.-M. (1997). Self-normalized large deviations. Ann. Probab. 25 285–328.
• Shao and Zhou (2012) Shao, Q. M. and Zhou, W. X. (2012). Cramér type moderate deviation theorems for Studentized non-linear statistics. Unpublished manuscript.
• Zhou (2007) Zhou, W. (2007). Asymptotic distribution of the largest off-diagonal entry of correlation matrices. Trans. Amer. Math. Soc. 359 5345–5363.