The Annals of Statistics

Posterior propriety and admissibility of hyperpriors in normal hierarchical models

James O. Berger, William Strawderman, and Dejun Tang

Full-text: Open access

Abstract

Hierarchical modeling is wonderful and here to stay, but hyperparameter priors are often chosen in a casual fashion. Unfortunately, as the number of hyperparameters grows, the effects of casual choices can multiply, leading to considerably inferior performance. As an extreme, but not uncommon, example use of the wrong hyperparameter priors can even lead to impropriety of the posterior.

For exchangeable hierarchical multivariate normal models, we first determine when a standard class of hierarchical priors results in proper or improper posteriors. We next determine which elements of this class lead to admissible estimators of the mean under quadratic loss; such considerations provide one useful guideline for choice among hierarchical priors. Finally, computational issues with the resulting posterior distributions are addressed.

Article information

Source
Ann. Statist., Volume 33, Number 2 (2005), 606-646.

Dates
First available in Project Euclid: 26 May 2005

Permanent link to this document
https://projecteuclid.org/euclid.aos/1117114331

Digital Object Identifier
doi:10.1214/009053605000000075

Mathematical Reviews number (MathSciNet)
MR2163154

Zentralblatt MATH identifier
1068.62005

Subjects
Primary: 62C15: Admissibility
Secondary: 62F15: Bayesian inference

Keywords
Covariance matrix quadratic loss frequentist risk posterior impropriety objective priors Markov chain Monte Carlo

Citation

Berger, James O.; Strawderman, William; Tang, Dejun. Posterior propriety and admissibility of hyperpriors in normal hierarchical models. Ann. Statist. 33 (2005), no. 2, 606--646. doi:10.1214/009053605000000075. https://projecteuclid.org/euclid.aos/1117114331


Export citation

References

  • Angers, J.-F. (1992). Use of the Student-$t$ prior for the estimation of normal means: A computational approach. In Bayesian Statistics 4 (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 567–575. Oxford Univ. Press, New York.
  • Angers, J.-F., MacGibbon, B. and Wang, S. (1998). Bayesian estimation of intra-block exchangeable normal means with applications. Sankhyā Ser. A 60 198–213.
  • Bayarri, M. J. and Berger, J. O. (2004). The interplay of Bayesian and frequentist analysis. Statist. Sci. 19 58–80.
  • Berger, J. O. (1980). A robust generalized Bayes estimator and confidence region for a multivariate normal mean. Ann. Statist. 8 716–761.
  • Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York.
  • Berger, J. O. and Robert, C. (1990). Subjective hierarchical Bayes estimation of a multivariate normal mean: On the frequentist interface. Ann. Statist. 18 617–651.
  • Berger, J. O. and Strawderman, W. (1996). Choice of hierarchical priors: Admissibility in estimation of normal means. Ann. Statist. 24 931–951.
  • Bernardo, J. M. and Smith, A. F. M. (1994). Bayesian Theory. Wiley, New York.
  • Brown, L. D. (1971). Admissible estimators, recurrent diffusions, and insoluble boundary value problems. Ann. Math. Statist. 42 855–903.
  • Brown, L. D. (2000). An essay on statistical decision theory. J. Amer. Statist. Assoc. 95 1277–1281.
  • Carlin, B. P. and Louis, T. A. (2000). Bayes and Empirical Bayes Methods for Data Analysis, 2nd ed. Chapman and Hall, Boca Raton, FL.
  • Chen, M.-H., Shao, Q.-M. and Ibrahim, J. G. (2000). Monte Carlo Methods in Bayesian Computation. Springer, New York.
  • Daniels, M. (1998). Computing posterior distributions for covariance matrices. In Computing Science and Statistics. Proc. 30th Symposium on the Interface 192–196. Interface, Fairfax Station, VA.
  • Daniels, M. and Kass, R. (1999). Nonconjugate Bayesian estimation of covariance matrices and its use in hierarchical models. J. Amer. Statist. Assoc. 94 1254–1263.
  • Daniels, M. and Pourahmadi, M. (2002). Bayesian analysis of covariance matrices and dynamic models for longitudinal data. Biometrika 89 553–566.
  • Efron, B. and Morris, C. (1972). Empirical Bayes on vector observations–-an extension of Stein's method. Biometrika 59 335–347.
  • Efron, B. and Morris, C. (1976). Multivariate empirical Bayes and estimation of covariance matrices. Ann. Statist. 4 22–32.
  • Everson, P. J. and Morris, C. (2000). Inference for multivariate normal hierarchical models. J. R. Stat. Soc. Ser. B Stat. Methodol. 62 399–412.
  • Farrell, R. H. (1985). Multivariate Calculation. Use of the Continuous Groups. Springer, New York.
  • Fourdrinier, D., Strawderman, W. and Wells, M. (1998). On the construction of Bayes minimax estimators. Ann. Statist. 26 660–671.
  • Ghosh, M. (1992). Hierarchical and empirical Bayes multivariate estimation. In Current Issues in Statistical Inference (M. Ghosh and P. K. Pathak, eds.) 151–177. IMS, Hayward, CA.
  • Haff, L. R. (1991). The variational form of certain Bayes estimators. Ann. Statist. 19 1163–1190.
  • Hobert, J. (2000). Hierarchical models: A current computational perspective. J. Amer. Statist. Assoc. 95 1312–1316.
  • Kotz, S., Balakrishnan, N. and Johnson, N. (2000). Continuous Multivariate Distributions 1. Models and Applications, 2nd ed. Wiley, New York.
  • Liechty, J. C., Liechty, M. W. and Müller, P. (2004). Bayesian correlation estimation. Biometrika 91 1–14.
  • Lin, S. P. and Perlman, M. D. (1985). A Monte Carlo comparison of four estimators of a covariance matrix. In Multivariate Analysis VI (P. R. Krishnaiah, ed.) 411–429. North-Holland, Amsterdam.
  • Loh, W.-L. (1991). Estimating covariance matrices. II. J. Multivariate Anal. 36 163–174.
  • Ni, S. and Sun, D. (2005). Bayesian estimate for vector autoregressive models. J. Bus. Econom. Statist. 23 105–117.
  • O'Hagan, A. (1990). On outliers and credence for location parameter inference. J. Amer. Statist. Assoc. 85 172–176.
  • Perron, F. (1992). Minimax estimators of a covariance matrix. J. Multivariate Anal. 43 16–28.
  • Robert, C. P. (2001). The Bayesian Choice, 2nd ed. Springer, New York.
  • Robert, C. P. and Casella, G. (1999). Monte Carlo Statistical Methods. Springer, New York.
  • Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proc. Third Berkeley Symp. Math. Statist. Probab. 1 197–206. Univ. California Press, Berkeley.
  • Stein, C. (1975). Estimation of a covariance matrix. Rietz Lecture, 39th Annual Meeting of the IMS. Atlanta, Georgia.
  • Strawderman, W. (1971). Proper Bayes minimax estimators of the multivariate normal mean. Ann. Math. Statist. 42 385–388.
  • Strawderman, W. (2000). Minimaxity. J. Amer. Statist. Assoc. 95 1364–1368.
  • Sugiura, N. and Ishibayashi, H. (1997). Reference prior Bayes estimator for bivariate normal covariance matrix with risk comparison. Comm. Statist. Theory Methods 26 2203–2221.
  • Sun, D., Tsutakawa, R. K. and He, Z. (2001). Propriety of posteriors with improper priors in hierarchical linear mixed models. Statist. Sinica 11 77–95.
  • Tang, D. (2001). Choice of priors for hierarchical models: Admissibility and computation. Ph.D. dissertation, Purdue Univ.
  • Yang, R. and Berger, J. O. (1994). Estimation of a covariance matrix using the reference prior. Ann. Statist. 22 1195–1211.