Institute of Mathematical Statistics Collections
The unbearable transparency of Stein estimation
Charles Stein  discovered that, under quadratic loss, the usual unbiased estimator for the mean vector of a multivariate normal distribution is inadmissible if the dimension n of the mean vector exceeds two. On the way, he constructed shrinkage estimators that dominate the usual estimator asymptotically in n. It has since been claimed that Stein’s results and the subsequent James–Stein estimator are counter-intuitive, even paradoxical, and not very useful. In response to such doubts, various authors have presented alternative derivations of Stein shrinkage estimators. Surely Stein himself did not find his results paradoxical. This paper argues that assertions of “paradoxical" or “counter-intuitive" or “not practical" have overlooked essential arguments and remarks in Stein’s beautifully written paper . Among these overlooked aspects are the asymptotic geometry of quadratic loss in high dimensions that makes Stein estimation transparent; the asymptotic optimality results that can be associate with Stein estimation; the explicit mention of practical multiple shrinkage estimators; and the foreshadowing of Stein confidence balls. These ideas are fundamental for studies of modern regularization estimators that rely on multiple shrinkage, whether implicitly or overtly.
First available in Project Euclid: 29 November 2010
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Copyright © 2010, Institute of Mathematical Statistics
Beran, Rudolf. The unbearable transparency of Stein estimation. Nonparametrics and Robustness in Modern Statistical Inference and Time Series Analysis: A Festschrift in honor of Professor Jana Jurečková, 25--34, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2010. doi:10.1214/10-IMSCOLL703. https://projecteuclid.org/euclid.imsc/1291044739
-  Beran, R. (1995). Stein confidence sets and the bootstrap. Statistica Sinica 5 109–127.
-  Beran, R. (1996). Stein estimation in high dimensions: a retrospective. In Madan Puri Festschrift (E. Brunner and M. Denker, eds.) 91–110. VSP, Zeist.
-  Beran, R. (2007). Adaptation over parametric families of symmetric linear estimators. Journal of Statistical Planning and Inference (Special Issue on Nonparametric Statistics and Related Topics) 137 684–696.
-  Beran, R. (2008). Estimating a mean matrix: boosting efficiency by multiple affine shrinkage. Annals of the Institute of Statistical Mathematics 60 843–864.
-  Beran, R. and Dümbgen, L. (1998). Modulation of estimators and confidence sets. Annals of Statistics 26 1826–1856.
-  Efron, B. and Morris, C. (1973). Stein’s estimation rule and its competitors — an empirical Bayes approach. Journal of the American Statistical Association 68 117–130.
-  Hasminski, R. Z. and Nussbaum, M. (1984). An asymptotic minimax bound in a regression problem with an increasing number of nuisance parameters. In Proceedings of the Third Prague Symposium on Asymptotic Statistics (P. Mandl and M. Hušková, eds.) 275–283. Elsevier, New York.
-  James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability (J. Neyman, ed.) 1 361–380. University of California Press.
-  Pinsker, M. S. (1980). Optimal filtration of square-integrable signals in Gaussian white noise. Problems of Information Transmission 16 120–133.
-  Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability (J. Neyman, ed.) 1 197–206. University of California Press.
-  Stein, C. (1966). An approach to the recovery of inter-block information in balanced incomplete block designs. In Festschrift for Jerzy Neyman (F. N. David, ed.) 351–364. Wiley, New York.
-  Stein, C. (1981) Estimation of the mean of a multivariate normal distribution. Annals of Statistics. 9 1135–1151.
-  Stigler, S. M. (1990). A Galtonian perspective on shrinkage estimators. Statistical Science 5 147–155.
-  Watson, G. S. (1983). Statistics on Spheres. Wiley-Interscience, New York.