Statistical Science

Shrinkage Estimation in Multilevel Normal Models

Carl N. Morris and Martin Lysy

Full-text: Open access


This review traces the evolution of theory that started when Charles Stein in 1955 [In Proc. 3rd Berkeley Sympos. Math. Statist. Probab. I (1956) 197–206, Univ. California Press] showed that using each separate sample mean from k ≥ 3 Normal populations to estimate its own population mean μi can be improved upon uniformly for every possible μ = (μ1,  …,  μk)'. The dominating estimators, referred to here as being “Model-I minimax,” can be found by shrinking the sample means toward any constant vector. Admissible minimax shrinkage estimators were derived by Stein and others as posterior means based on a random effects model, “Model-II” here, wherein the μi values have their own distributions. Section 2 centers on Figure 2, which organizes a wide class of priors on the unknown Level-II hyperparameters that have been proved to yield admissible Model-I minimax shrinkage estimators in the “equal variance case.” Putting a flat prior on the Level-II variance is unique in this class for its scale-invariance and for its conjugacy, and it induces Stein’s harmonic prior (SHP) on μi.

Component estimators with real data, however, often have substantially “unequal variances.” While Model-I minimaxity is achievable in such cases, this standard requires estimators to have “reverse shrinkages,” as when the large variance component sample means shrink less (not more) than the more accurate ones. Section 3 explains how Model-II provides appropriate shrinkage patterns, and investigates especially estimators determined exactly or approximately from the posterior distributions based on the objective priors that produce Model-I minimaxity in the equal variances case. While correcting the reversed shrinkage defect, Model-II minimaxity can hold for every component. In a real example of hospital profiling data, the SHP prior is shown to provide estimators that are Model-II minimax, and posterior intervals that have adequate Model-II coverage, that is, both conditionally on every possible Level-II hyperparameter and for every individual component μi, i = 1,  …,  k.

Article information

Statist. Sci., Volume 27, Number 1 (2012), 115-134.

First available in Project Euclid: 14 March 2012

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Hierarchical model empirical Bayes unequal variances Model-II evaluations Stein’s harmonic prior


Morris, Carl N.; Lysy, Martin. Shrinkage Estimation in Multilevel Normal Models. Statist. Sci. 27 (2012), no. 1, 115--134. doi:10.1214/11-STS363.

Export citation


  • [1] Baranchik, A. (1964). Multiple regression and estimation of the mean of a multivariate normal population Technical Report 51, Dept. Statistics, Stanford Univ.
  • [2] Berger, J. O. (1976). Admissible minimax estimation of a multivariate normal mean with arbitrary quadratic loss. Ann. Statist. 4 223–226.
  • [3] Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. ed. Springer, New York.
  • [4] Brown, L. D. (1966). On the admissibility of invariant estimators of one or more location parameters. Ann. Math. Statist. 37 1087–1136.
  • [5] Brown, L. D. (2009). Personal communication.
  • [6] Brown, L. D., Nie, H. and Xie, X. (2011). Ensemble minimax estimation for multivariate normal means. Ann. Statist. To appear.
  • [7] Christiansen, C. L. and Morris, C. N. (1997). Hierarchical Poisson regression modeling. J. Amer. Statist. Assoc. 92 618–632.
  • [8] Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential families. Ann. Statist. 7 269–281.
  • [9] Efron, B. and Morris, C. (1975). Data analysis using Stein’s estimator and its generalizations. J. Amer. Statist. Assoc. 70 311–319.
  • [10] Efron, B. and Morris, C. (1976). Families of minimax estimators of the mean of a multivariate normal distribution. Ann. Statist. 4 11–21.
  • [11] Everson, P. J. and Morris, C. N. (2000). Inference for multivariate normal hierarchical models. J. R. Stat. Soc. Ser. B Stat. Methodol. 62 399–412.
  • [12] Hudson, H. M. (1974). Empirical Bayes estimation Technical Report 58, Dept. Statistics, Stanford Univ.
  • [13] Jackson, D. A., O’Donovan, T. M., Zimmer, W. J. and Deely, J. J. (1970). $\mathcal{G}_{2}$-minimax estimators in the exponential family. Biometrika 57 439–443.
  • [14] James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4th Berkeley Sympos. Math. Statist. Probab. I 361–379. Univ. California Press, Berkeley, CA.
  • [15] Li, H. and Lahiri, P. (2010). An adjusted maximum likelihood method for solving small area estimation problems. J. Multivariate Anal. 101 882–892.
  • [16] Morris, C. (1977). Interval estimation for empirical Bayes generalizations of Stein’s estimator. The Rand Paper Series, The Rand Corporation.
  • [17] Morris, C. N. (1983). Parametric empirical Bayes inference: Theory and applications (with discussion). J. Amer. Statist. Assoc. 78 47–65.
  • [18] Morris, C. N. (1983). Natural exponential families with quadratic variance functions: Statistical theory. Ann. Statist. 11 515–529.
  • [19] Morris, C. N. (1983). Parametric empirical Bayes confidence intervals. In Scientific Inference, Data Analysis, and Robustness (Madison, Wis., 1981). Publ. Math. Res. Center Univ. Wisconsin 48 25–50. Academic Press, Orlando, FL.
  • [20] Morris, C. N. (1988). Approximating posterior distributions and posterior moments. In Bayesian Statistics 3 (Valencia, 1987) 327–344. Oxford Univ. Press, New York.
  • [21] Morris, C. N. and Lock, K. F. (2009). Unifying the named natural exponential families and their relatives. Amer. Statist. 63 247–253.
  • [22] Morris, C. and Tang, R. (2011). Estimating random effects via adjustment for density maximization. Statist. Sci. 26 271–287.
  • [23] Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proc. 3rd Berkeley Sympos. Math. Statist. Probab. I 197–206. Univ. California Press, Berkeley.
  • [24] Stein, C. (1966). An approach to the recovery of inter-block information in balanced incomplete block designs. In Research Papers in Statistics (Festschrift J. Neyman) 351–366. Wiley, London.
  • [25] Stein, C. (1974). Estimation of the mean of a multivariate normal distribution. In Proceedings of Prague Symposium on Asymptotic Statistics (Charles Univ., Prague, 1973) II 345–381. Charles Univ., Prague.
  • [26] Stein, C. M. (1962). Confidence sets for the mean of a multivariate normal distribution. J. Roy. Statist. Soc. Ser. B 24 265–296.
  • [27] Stein, C. M. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135–1151.
  • [28] Strawderman, W. E. (1971). Proper Bayes minimax estimators of the multivariate normal mean. Ann. Math. Statist. 42 385–388.