The Annals of Statistics

Improved multivariate normal mean estimation with unknown covariance when $p$ is greater than $n$

Didier Chételat and Martin T. Wells

Full-text: Open access


We consider the problem of estimating the mean vector of a $p$-variate normal $(\theta,\Sigma)$ distribution under invariant quadratic loss, $(\delta-\theta)'\Sigma^{-1}(\delta-\theta)$, when the covariance is unknown. We propose a new class of estimators that dominate the usual estimator $\delta^{0}(X)=X$. The proposed estimators of $\theta$ depend upon $X$ and an independent Wishart matrix $S$ with $n$ degrees of freedom, however, $S$ is singular almost surely when $p>n$. The proof of domination involves the development of some new unbiased estimators of risk for the $p>n$ setting. We also find some relationships between the amount of domination and the magnitudes of $n$ and $p$.

Article information

Ann. Statist., Volume 40, Number 6 (2012), 3137-3160.

First available in Project Euclid: 22 February 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F10: Point estimation
Secondary: 62C20: Minimax procedures 62H12: Estimation

Covariance estimation James–Stein estimation invariant quadratic loss large-$p$–small-$n$ problems location parameter minimax estimation Moore–Penrose inverse risk function singular Wishart distribution


Chételat, Didier; Wells, Martin T. Improved multivariate normal mean estimation with unknown covariance when $p$ is greater than $n$. Ann. Statist. 40 (2012), no. 6, 3137--3160. doi:10.1214/12-AOS1067.

Export citation


  • Baranchik, A. J. (1970). A family of minimax estimators of the mean of a multivariate normal distribution. Ann. Math. Statist. 41 642–645.
  • Berger, J. O. and Bock, M. E. (1976). Combining independent normal mean estimation problems with unknown variances. Ann. Statist. 4 642–648.
  • Berger, J. and Haff, L. R. (1983). A class of minimax estimators of a normal mean vector for arbitrary quadratic loss and unknown covariance matrix. Statist. Decisions 1 105–129.
  • Berger, J., Bock, M. E., Brown, L. D., Casella, G. and Gleser, L. (1977). Minimax estimation of a normal mean vector for arbitrary quadratic loss and unknown covariance matrix. Ann. Statist. 5 763–771.
  • Bickel, P. J. and Levina, E. (2008). Regularized estimation of large covariance matrices. Ann. Statist. 36 199–227.
  • Brown, L. D. (1975). Estimation with incompletely specified loss functions (the case of several location parameters). J. Amer. Statist. Assoc. 70 417–427.
  • Brown, L. D., Nie, H. and Xie, X. (2013). Ensemble minimax estimation for multivariate normal means. Ann. Statist. To appear.
  • Casella, G. (1980). Minimax ridge regression estimation. Ann. Statist. 8 1036–1056.
  • d’Aspremont, A., Banerjee, O. and El Ghaoui, L. (2008). First-order methods for sparse covariance selection. SIAM J. Matrix Anal. Appl. 30 56–66.
  • Fang, K. T., Kotz, S. and Ng, K. W. (1990). Symmetric Multivariate and Related Distributions. Monographs on Statistics and Applied Probability 36. Chapman & Hall, London.
  • Fourdrinier, D. and Strawderman, W. E. (2003). On Bayes and unbiased estimators of loss. Ann. Inst. Statist. Math. 55 803–816.
  • Fourdrinier, D., Strawderman, W. E. and Wells, M. T. (2003). Robust shrinkage estimation for elliptically symmetric distributions with unknown covariance matrix. J. Multivariate Anal. 85 24–39.
  • Gleser, L. J. (1979). Minimax estimation of a normal mean vector when the covariance matrix is unknown. Ann. Statist. 7 838–846.
  • Gleser, L. J. (1986). Minimax estimators of a normal mean vector for arbitrary quadratic loss and unknown covariance matrix. Ann. Statist. 14 1625–1633.
  • Golub, G. H. and Pereyra, V. (1973). The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM J. Numer. Anal. 10 413–432.
  • Haff, L. R. (1979). An identity for the Wishart distribution with applications. J. Multivariate Anal. 9 531–544.
  • James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I 361–379. Univ. California Press, Berkeley, CA.
  • Konno, Y. (2009). Shrinkage estimators for large covariance matrices in multivariate real and complex normal distributions under an invariant quadratic loss. J. Multivariate Anal. 100 2237–2253.
  • Kubokawa, T. and Srivastava, M. S. (2001). Robust improvement in estimation of a mean matrix in an elliptically contoured distribution. J. Multivariate Anal. 76 138–152.
  • Kubokawa, T. and Srivastava, M. S. (2008). Estimation of the precision matrix of a singular Wishart distribution and its application in high-dimensional data. J. Multivariate Anal. 99 1906–1928.
  • Ledoit, O. and Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88 365–411.
  • Levina, E., Rothman, A. and Zhu, J. (2008). Sparse estimation of large covariance matrices via a nested Lasso penalty. Ann. Appl. Stat. 2 245–263.
  • Magnus, J. R. (1990). On certain moments relating to ratios of quadratic forms in normal variables: Further results. Sankhyā Ser. B 52 1–13.
  • Morris, C. and Lysy, M. (2012). Shrinkage estimation in multi-level normal models. Statist. Sci. 27 115–134.
  • Muirhead, R. J. (1982). Aspects of Multivariate Statistical Theory. Wiley, New York.
  • Rothman, A. J., Bickel, P. J., Levina, E. and Zhu, J. (2008). Sparse permutation invariant covariance estimation. Electron. J. Stat. 2 494–515.
  • Srivastava, M. S. (2003). Singular Wishart and multivariate beta distributions. Ann. Statist. 31 1537–1560.
  • Srivastava, M. S. (2007). Multivariate theory for analyzing high dimensional data. J. Japan Statist. Soc. 37 53–86.
  • Srivastava, M. S. and Bilodeau, M. (1989). Stein estimation under elliptical distributions. J. Multivariate Anal. 28 247–259.
  • Srivastava, M. S. and Fujikoshi, Y. (2006). Multivariate analysis of variance with fewer observations than the dimension. J. Multivariate Anal. 97 1927–1940.
  • Srivastava, M. S. and Khatri, C. G. (1979). An Introduction to Multivariate Statistics. North-Holland, New York.
  • Stein, C. M. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135–1151.
  • Stein, C. (1977). Lectures on the theory of estimation of many parameters. In Studies in the Statistical Theory of Estimation, Part I 74 4–65. Proc. Scientific Seminars Steklov Institute, Leningrad Division. (In Russian).
  • Tian, Y. and Cheng, S. (2004). Some identities for Moore–Penrose inverses of matrix products. Linear Multilinear Algebra 52 405–420.