Open Access
2023 Envelopes and principal component regression
Xin Zhang, Kai Deng, Qing Mai
Author Affiliations +
Electron. J. Statist. 17(2): 2447-2484 (2023). DOI: 10.1214/23-EJS2154


Envelope methods offer targeted dimension reduction for various statistical models. The goal is to improve efficiency in multivariate parameter estimation by projecting the data onto a lower-dimensional subspace known as the envelope. Envelope approaches have advantages in analyzing data with highly correlated variables, but their iterative Grassmannian optimization algorithms do not scale very well with high-dimensional data. While the connections between envelopes and partial least squares in multivariate linear regression have promoted recent progress in high-dimensional studies of envelopes, we propose a more straightforward way of envelope modeling from a new principal component regression perspective. The proposed procedure, Non-Iterative Envelope Component Estimation (NIECE), has excellent computational advantages over the iterative Grassmannian optimization alternatives in high dimensions. We develop a unified theory that bridges the gap between envelope methods and principal components in regression. The new theoretical insights also shed light on the envelope subspace estimation error as a function of eigenvalue gaps of two symmetric positive definite matrices used in envelope modeling. We apply the new theory and algorithm to several envelope models, including response and predictor reduction in multivariate linear models, logistic regression, and Cox proportional hazard model. Simulations and illustrative data analysis show the potential for NIECE to improve standard methods in linear and generalized linear models significantly.

Funding Statement

Research is partly supported by grants CCF-1908969, DMS-2053697, and DMS-2113590 from the US National Science Foundation.


Download Citation

Xin Zhang. Kai Deng. Qing Mai. "Envelopes and principal component regression." Electron. J. Statist. 17 (2) 2447 - 2484, 2023.


Received: 1 September 2022; Published: 2023
First available in Project Euclid: 10 October 2023

MathSciNet: MR4652861
Digital Object Identifier: 10.1214/23-EJS2154

Primary: 62H25
Secondary: 62J12

Keywords: envelope methods , penalized matrix decomposition , principal component , sufficient dimension reduction

Vol.17 • No. 2 • 2023
Back to Top