Open Access
October 2020 Permutation methods for factor analysis and PCA
Edgar Dobriban
Ann. Statist. 48(5): 2824-2847 (October 2020). DOI: 10.1214/19-AOS1907


Researchers often have datasets measuring features $x_{ij}$ of samples, such as test scores of students. In factor analysis and PCA, these features are thought to be influenced by unobserved factors, such as skills. Can we determine how many components affect the data? This is an important problem, because decisions made here have a large impact on all downstream data analysis. Consequently, many approaches have been developed. Parallel Analysis is a popular permutation method: it randomly scrambles each feature of the data. It selects components if their singular values are larger than those of the permuted data. Despite widespread use, as well as empirical evidence for its accuracy, it currently has no theoretical justification.

In this paper, we show that parallel analysis (or permutation methods) consistently select the large components in certain high-dimensional factor models. However, when the signals are too large, the smaller components are not selected. The intuition is that permutations keep the noise invariant, while “destroying” the low-rank signal. This provides justification for permutation methods. Our work also uncovers drawbacks of permutation methods, and paves the way to improvements.


Download Citation

Edgar Dobriban. "Permutation methods for factor analysis and PCA." Ann. Statist. 48 (5) 2824 - 2847, October 2020.


Received: 1 May 2018; Revised: 1 September 2019; Published: October 2020
First available in Project Euclid: 19 September 2020

MathSciNet: MR4152122
Digital Object Identifier: 10.1214/19-AOS1907

Primary: 62H25
Secondary: 62H12

Keywords: factor analysis , high-dimensional asymptotics , parallel analysis , PCA , permutation methods

Rights: Copyright © 2020 Institute of Mathematical Statistics

Vol.48 • No. 5 • October 2020
Back to Top