The Annals of Mathematical Statistics

An Optimal Property of Principal Components

J. N. Darroch

Full-text: Open access


Let $x' = (x_1, x_2, \cdots, x_p)$ be a random vector and let $E\lbrack x\rbrack = 0, E\lbrack xx'\rbrack = \Sigma = (\sigma_{ij})$ where we assume that $\Sigma$ is non-singular. Further let \[ \Sigma = T\Lambda T' = (t_1, t_2, \cdots, t_p) \left[ \begin{array}{c|c|c|c|c|c|c}\lambda_1 & 0 & \cdot & \cdot & \cdot & 0 \\ 0 & \lambda_2 & \cdot & \cdot & \cdot & 0 \\ \vdots & \vdots & \vdots & \vdots \\0 & 0 & \cdot & \cdot & \cdot & \lambda_p\end{array} \right] \] \[ \left[ \begin{array}{|c}t'_1 \\ t'_2 \\ \vdots \\ t'_p\end{array}\right] \] where $TT' = I$ and where we suppose that the eigenvalues of $\Sigma$ are in order of decreasing magnitude, that is $\lambda_1 \geqq \lambda_2 \geqq \cdots \geqq \lambda_p > 0$. The principal components of $x$, namely $u_1 = t'_1 x, u_2 = t'_2 x, \cdots, u_p = t_p' x$ were introduced by Hotelling (1933) who characterised them by certain optimal properties. Since then Girshick (1936), Anderson (1958) and Kullback (1959) have characterised the principal components by slightly different sets of optimal properties. Thus Anderson shows that $u_1$ is the linear function $\alpha_1' x$ having maximum variance subject to $\alpha_1' \alpha_1 = 1; u_2$ is the linear function $\alpha_2'x$ which is uncorrelated with $u_1$ and has maximum variance subject to $\alpha_2'\alpha_2 = 1$; and so on. The above mentioned characterisations have two properties in common; they introduce the principal components one by one and, more importantly, the optimal properties hold only with-in the class of linear functions of $x_1, x_2, \cdots, x_p$. In the following theorem the first $k$ principal components are characterized by an optimal property within the class of all random variables.

Article information

Ann. Math. Statist., Volume 36, Number 5 (1965), 1579-1582.

First available in Project Euclid: 27 April 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier



Darroch, J. N. An Optimal Property of Principal Components. Ann. Math. Statist. 36 (1965), no. 5, 1579--1582. doi:10.1214/aoms/1177699920.

Export citation