Institute of Mathematical Statistics Collections

The unbearable transparency of Stein estimation

Rudolf Beran

Full-text: Open access

Abstract

Charles Stein [10] discovered that, under quadratic loss, the usual unbiased estimator for the mean vector of a multivariate normal distribution is inadmissible if the dimension n of the mean vector exceeds two. On the way, he constructed shrinkage estimators that dominate the usual estimator asymptotically in n. It has since been claimed that Stein’s results and the subsequent James–Stein estimator are counter-intuitive, even paradoxical, and not very useful. In response to such doubts, various authors have presented alternative derivations of Stein shrinkage estimators. Surely Stein himself did not find his results paradoxical. This paper argues that assertions of “paradoxical" or “counter-intuitive" or “not practical" have overlooked essential arguments and remarks in Stein’s beautifully written paper [10]. Among these overlooked aspects are the asymptotic geometry of quadratic loss in high dimensions that makes Stein estimation transparent; the asymptotic optimality results that can be associate with Stein estimation; the explicit mention of practical multiple shrinkage estimators; and the foreshadowing of Stein confidence balls. These ideas are fundamental for studies of modern regularization estimators that rely on multiple shrinkage, whether implicitly or overtly.

Chapter information

Source
J. Antoch, M. Hušková and P.K. Sen, eds., Nonparametrics and Robustness in Modern Statistical Inference and Time Series Analysis: A Festschrift in honor of Professor Jana Jurečková (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2010), 25-34

Dates
First available in Project Euclid: 29 November 2010

Permanent link to this document
https://projecteuclid.org/euclid.imsc/1291044739

Digital Object Identifier
doi:10.1214/10-IMSCOLL703

Mathematical Reviews number (MathSciNet)
MR2808363

Subjects
Primary: 62F12: Asymptotic properties of estimators 62J07: Ridge regression; shrinkage estimators
Secondary: 62-02: Research exposition (monographs, survey articles)

Keywords
dimensional asymptotics orthogonal equivariance

Rights
Copyright © 2010, Institute of Mathematical Statistics

Citation

Beran, Rudolf. The unbearable transparency of Stein estimation. Nonparametrics and Robustness in Modern Statistical Inference and Time Series Analysis: A Festschrift in honor of Professor Jana Jurečková, 25--34, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2010. doi:10.1214/10-IMSCOLL703. https://projecteuclid.org/euclid.imsc/1291044739


Export citation

References

  • [1] Beran, R. (1995). Stein confidence sets and the bootstrap. Statistica Sinica 5 109–127.
  • [2] Beran, R. (1996). Stein estimation in high dimensions: a retrospective. In Madan Puri Festschrift (E. Brunner and M. Denker, eds.) 91–110. VSP, Zeist.
  • [3] Beran, R. (2007). Adaptation over parametric families of symmetric linear estimators. Journal of Statistical Planning and Inference (Special Issue on Nonparametric Statistics and Related Topics) 137 684–696.
  • [4] Beran, R. (2008). Estimating a mean matrix: boosting efficiency by multiple affine shrinkage. Annals of the Institute of Statistical Mathematics 60 843–864.
  • [5] Beran, R. and Dümbgen, L. (1998). Modulation of estimators and confidence sets. Annals of Statistics 26 1826–1856.
  • [6] Efron, B. and Morris, C. (1973). Stein’s estimation rule and its competitors — an empirical Bayes approach. Journal of the American Statistical Association 68 117–130.
  • [7] Hasminski, R. Z. and Nussbaum, M. (1984). An asymptotic minimax bound in a regression problem with an increasing number of nuisance parameters. In Proceedings of the Third Prague Symposium on Asymptotic Statistics (P. Mandl and M. Hušková, eds.) 275–283. Elsevier, New York.
  • [8] James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability (J. Neyman, ed.) 1 361–380. University of California Press.
  • [9] Pinsker, M. S. (1980). Optimal filtration of square-integrable signals in Gaussian white noise. Problems of Information Transmission 16 120–133.
  • [10] Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability (J. Neyman, ed.) 1 197–206. University of California Press.
  • [11] Stein, C. (1966). An approach to the recovery of inter-block information in balanced incomplete block designs. In Festschrift for Jerzy Neyman (F. N. David, ed.) 351–364. Wiley, New York.
  • [12] Stein, C. (1981) Estimation of the mean of a multivariate normal distribution. Annals of Statistics. 9 1135–1151.
  • [13] Stigler, S. M. (1990). A Galtonian perspective on shrinkage estimators. Statistical Science 5 147–155.
  • [14] Watson, G. S. (1983). Statistics on Spheres. Wiley-Interscience, New York.