## Bayesian Analysis

### A comparison of Bayesian and likelihood-based methods for fitting multilevel models

#### Abstract

We use simulation studies, whose design is realistic for educational and medical research (as well as other fields of inquiry), to compare Bayesian and likelihood-based methods for fitting variance-components (VC) and random-effects logistic regression (RELR) models. The likelihood (and approximate likelihood) approaches we examine are based on the methods most widely used in current applied multilevel (hierarchical) analyses: maximum likelihood (ML) and restricted ML (REML) for Gaussian outcomes, and marginal and penalized quasi-likelihood (MQL and PQL) for Bernoulli outcomes. Our Bayesian methods use Markov chain Monte Carlo (MCMC) estimation, with adaptive hybrid Metropolis-Gibbs sampling for RELR models, and several diffuse prior distributions ($\Gamma^{ -1 }( \epsilon, \epsilon )$ and $U( 0, \frac{ 1 }{ \epsilon } )$ priors for variance components). For evaluation criteria we consider bias of point estimates and nominal versus actual coverage of interval estimates in repeated sampling. In two-level VC models we find that (a) both likelihood-based and Bayesian approaches can be made to produce approximately unbiased estimates, although the automatic manner in which REML accomplishes this is an advantage, but (b) both approaches had difficulty achieving nominal coverage in small samples and with small values of the intraclass correlation. With the three-level RELR models we examine we find that (c) quasi-likelihood methods for estimating random-effects variances perform badly with respect to bias and coverage in the example we simulated, and (d) Bayesian diffuse-prior methods lead to well-calibrated point and interval RELR estimates. While it is true that the likelihood-based methods we study are considerably faster computationally than MCMC, (i) steady improvements in recent years in both hardware speed and efficiency of Monte Carlo algorithms and (ii) the lack of calibration of likelihood-based methods in some common hierarchical settings combine to make MCMC-based Bayesian fitting of multilevel models an attractive approach, even with rather large data sets. Other analytic strategies based on less approximate likelihood methods are also possible but would benefit from further study of the type summarized here.

#### Article information

Source
Bayesian Anal., Volume 1, Number 3 (2006), 473-514.

Dates
First available in Project Euclid: 22 June 2012

https://projecteuclid.org/euclid.ba/1340371047

Digital Object Identifier
doi:10.1214/06-BA117

Mathematical Reviews number (MathSciNet)
MR2221283

Zentralblatt MATH identifier
1331.62125

#### Citation

Browne, William J.; Draper, David. A comparison of Bayesian and likelihood-based methods for fitting multilevel models. Bayesian Anal. 1 (2006), no. 3, 473--514. doi:10.1214/06-BA117. https://projecteuclid.org/euclid.ba/1340371047

#### References

• Aitkin, M. 1996. A general maximum likelihood analysis of overdispersion in generalized linear models. Statistics and Computing 6: 251–262.
• –-. 1999. A general maximum likelihood analysis of variance components in generalized linear models. Biometrics 55: 117–128.
• –-. 1999. Meta-analysis by random-effects modelling in generalized linear models. Statistics in Medicine 18: 2343–2351.
• Bernardo, J. M. and A. F. M. Smith. 1994. Bayesian Theory. New York: Wiley.
• Besag, J., P. Green, D. Higdon, and K. Mengersen. 1995. Bayesian computation and stochastic systems (with discussion). Statistical Science 10: 3–41.
• Box, G. E. P. and G. C. Tiao. 1973. Bayesian Inference in Statistical Analysis. New York: Wiley.
• Breslow, N. E. and D. G. Clayton. 1993. Approximate inference in generalized linear mixed models. Journal of Statistical Computation and Simulation 88: 9–25.
• Brown, K. G. and M. A. Burgess. 1984. On maximum likelihood and restricted maximum likelihood approaches to estimation of variance components. Journal of Statistical Computation and Simulation 19: 59–77.
• Browne, W. J. 1998. Applying MCMC Methods to Multilevel Models. Ph.D. dissertation, Department of Mathematical Sciences, University of Bath, U.K.
• Browne, W. J. and D. Draper. 2000. Implementation and performance issues in the Bayesian fitting of multilevel models. Computational Statistics 15: 391–420.
• Browne, W. J., D. Draper, H. Goldstein, and J. Rasbash. 2002. Bayesian and likelihood methods for fitting multilevel models with complex level–1 variation. Computational Statistics and Data Analysis 39: 203–225.
• Bryk, A. S. and S. W. Raudenbush. 1992. Hierarchical Linear Models: Applications and Data Analysis Methods. London: Sage.
• Carlin, B. 1992. Discussion of “Hierarchical models for combining information and for meta-analysis," by C. N. Morris and S. L. Normand. In Bayesian Statistics, vol. 4, J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith (editors), 336–338. Oxford: Clarendon Press.
• Carlin, B. and T. A. Louis. 2001. Bayes and Empirical Bayes Methods for Data Analysis. 2nd ed. London: Chapman & Hall.
• Chaloner, K. 1987. A Bayesian approach to the estimation of variance components in the unbalanced one-way random-effects model. Technometrics 29: 323–337.
• Cochran, W. G. 1977. Sampling Techniques. 3rd ed. New York: Wiley.
• Corbeil, R. R. and S. R. Searle. 1976. Restricted maximum likelihood (REML) estimation of variance components in mixed models. Technometrics 18: 31–38.
• Daniels, M. 1999. A prior for the variance in hierarchical models. Canadian Journal of Statistics 27: 569–580.
• Daniels, M. J. and C. Gatsonis. 1999. Hierarchical generalized linear models in the analysis of variations in health care utilization. Journal of the American Statistical Association 94: 29–42.
• Dawid, A. P. 1985. Calibration-based empirical probability. Annals of Statistics 13: 1251–1274.
• Donner, A. 1986. A review of inference procedures for the intraclass correlation coefficient in the one-way random effects model. International Statistical Review 54: 67–82.
• Draper, D. 1995. Inference and hierarchical modeling in the social sciences (with discussion). Journal of Educational and Behavioral Statistics 20: 115–147, 233–239.
• DuMouchel, W. 1990. Bayesian meta-analysis. In Statistical Methodology in the Pharmaceutical Sciences, D. Berry (editor), 509–529. New York: Marcel Dekker.
• DuMouchel, W. and C. Waternaux. 1992. Discussion of “Hierarchical models for combining information and for meta-analysis," by C. N. Morris and S. L. Normand. In Bayesian Statistics, vol. 4, J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith (editors), 338–341. Oxford: Clarendon Press.
• Engel, B. 1998. A simple illustration of the failure of PQL, IRREML and APHL as approximate ML methods for mixed models for binary data. Biometrical Journal 40: 141–154.
• Freedman, D. 1999. On the Bernstein-von Mises theorem with infinite-dimensional parameters. Annals of Statistics 27: 1119–1140.
• Gelfand, A. and A. F. M. Smith. 1990. Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association 85: 398–409.
• Gelfand, A. E., S. K. Sahu, and B. P. Carlin. 1995. Efficient parameterizations for generalized linear mixed models (with discussion). In Bayesian Statistics, vol. 5, J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith (editors), 165–180. Oxford: Clarendon Press.
• Gelman, A. 2006. Prior distributions for variance parameters in hierarchical models. Bayesian Analysis (this issue).
• Gelman, A., J. B. Carlin, H. S. Stern, and D. B. Rubin. 2003. Bayesian Data Analysis. 2nd ed. London: Chapman & Hall.
• Gelman, A., G. O. Roberts, and W. R. Gilks. 1995. Efficient Metropolis jumping rules. In Bayesian Statistics, vol. 5, J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith (editors), 599–607. Oxford: Clarendon Press.
• Gelman, A. and D. B. Rubin. 1992. Inference from iterative simulation using multiple sequences (with discussion). Statistical Science 7: 457–511.
• Gilks, W. R., S. Richardson, and D. J. Spiegelhalter. 1996. Markov Chain Monte Carlo in Practice. London: Chapman & Hall.
• Gilks, W. R., G. O. Roberts, and S. K. Sahu. 1998. Adaptive Markov chain Monte Carlo sampling through regeneration. Journal of the American Statistical Association 93: 1045–1054.
• Gilks, W. R. and P. Wild. 1992. Adaptive rejection sampling for Gibbs sampling. Applied Statistics 41: 337–348.
• Goldstein, H. 1986. Multilevel mixed linear model analysis using iterative generalised least squares. Biometrika 73: 43–56.
• –-. 1989. Restricted unbiased iterative generalised least squares estimation. Biometrika 76: 622–623.
• –-. 2002. Multilevel Statistical Models. 3rd ed. London: Hodder Arnold.
• Goldstein, H. and J. Rasbash. 1996. Improved approximations for multilevel models with binary responses. Journal of the Royal Statistical Society (Series A) 159: 505–513.
• Goldstein, H., J. Rasbash, M. Yang, G. Woodhouse, H. Pan, D. Nutall, and S. Thomas. 1993. A multilevel analysis of school examination results. Oxford Review of Education 19: 425–433.
• Goldstein, H. and D. J. Spiegelhalter. 1996. League tables and their limitations: statistical issues in comparisons of institutional performance (with discussion). Journal of the Royal Statistical Society (Series A) 159: 385–444.
• Harville, D. A. and A. G. Zimmermann. 1996. The posterior distribution of the fixed and random effects in a mixed-effects linear model. Journal of Statistical Computation and Simulation 54: 211–229.
• Henderson, C. R. 1950. Estimation of genetic parameters (abstract). Annals of Mathematical Statistics 21: 309–310.
• Huber, D. A., T. L. White, and G. R. Hodge. 1994. Variance-component estimation techniques compared for two mating designs with forest genetic architecture through computer simulation. Theoretical and Applied Genetics 88: 236–242.
• Huber, P. J. 1967. The behavior of maximum likelihood estimates under non-standard conditions. In Proceedings of the Fifth Berkeley Symposium in Mathematical Statistics and Probability, vol. 1, 221–233. Berkeley CA: University of California Press.
• Hulting, F. L. and D. A. Harville. 1991. Some Bayesian and non-Bayesian procedures for the analysis of comparative experiments and for small-area estimation: computational aspects, frequentist properties, and relationships. Journal of the American Statistical Association 86: 557–568.
• Johnson, N. L., S. Kotz, and N. Balakrishnan. 1994. Continuous Univariate Distributions (Volume 1). 2nd ed. New York: Wiley.
• Kahn, M. J. and A. E. Raftery. 1996. Discharge rates of Medicare stroke patients to skilled nursing facilities: Bayesian logistic regression with unobserved heterogeneity. Journal of the American Statistical Association 91: 29–41.
• Kass, R. E. and D. Steffey. 1989. Approximate Bayesian inference in conditionally independent hierarchical models (parametric empirical Bayes models). Journal of the American Statistical Association 84: 717–726.
• Kass, R. E. and L. Wasserman. 1996. The selection of prior distributions by formal rules. Journal of the American Statistical Association 91: 1343–1370.
• Klotz, J. H., R. C. Milton, and S. Zacks. 1969. Mean square efficiency of estimators of variance components. Journal of the American Statistical Association 64: 1383–1394.
• Lee, Y. and J. A. Nelder. 1996. Hierarchical generalized linear models (with discussion). Journal of the Royal Statistical Society (Series B) 58: 619–678.
• –-. 2001. Hierarchical generalized linear models: a synthesis of generalized linear models, random-effect models, and structured dispersion. Biometrika 88: 987–1006.
• Lesaffre, E. and B. Spiessens. 2001. On the effect of the number of quadrature points in a logistic random-effects model: an example. Journal of the Royal Statistical Society (Series C) 50: 325–335.
• Liu, J. N. and J. S. Hodges. 2003. Posterior bimodality in the balanced one-way random-effects model. Journal of the Royal Statistical Society (Series B) 65: 247–255.
• Longford, N. T. 1987. A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects. Biometrika 74: 817–827.
• –-. 1997. Comment on “Improved approximations for multilevel models with binary responses," by H. Goldstein and J. Rasbash. Journal of the Royal Statistical Society (Series A) 160: 593.
• –-. 2000. On estimating standard errors in multilevel analysis. The Statistician 49: 389–398.
• Mortimore, P., P. Sammons, L. Stoll, D. Lewis, and R. Ecob. 1988. School Matters. Wells: Open Books.
• Müller, P. 1993. A generic approach to posterior integration and Gibbs sampling. Technical report, Institute of Statistics and Decision Sciences, Duke University.
• Natarajan, R. and R. E. Kass. 2000. Reference Bayesian methods for generalized linear mixed models. Journal of the American Statistical Association 95: 227–237.
• –-. 2006. A defaulty conjugate prior for variance components in generalized linear mixed models. Bayesian Analysis (this issue).
• Pebley, A. R. and N. Goldman. 1992. Family, community, ethnic identity, and the use of formal health care services in Guatemala. Working Paper 92–12. Princeton NJ: Office of Population Research.
• Pinheiro, J. C. and D. M. Bates. 1995. Approximations to the log-likelihood function in the non-linear mixed-effects model. Journal of Computational and Graphical Statistics 4: 12–35.
• Portnoy, S. 1971. Formal Bayes estimation with application to a random-effects model. Annals of Mathematical Statistics 42: 1379–1388.
• Raftery, A. L. and S. Lewis. 1992. How many iterations in the Gibbs sampler? In Bayesian Statistics, vol. 4, J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith (editors), 763–774. Oxford: Clarendon Press.
• Rasbash, J., F. Steele, W. Browne, and B. Prosser. 2005. A User's Guide To MLwiN, Version 2.0. University of Bristol, U.K.
• Raudenbush, S., A. Bryk, and R. Congdon. 2005. HLM: Hierarchical Linear and Nonlinear Modeling. Available at www.ssicentral.com/hlm.
• Raudenbush, S. W. 1994. Equivalence of Fisher scoring to iterative generalized least squares in the normal case, with application to hierarchical linear models. Technical Report, College of Education, Michigan State University.
• Raudenbush, S. W., M.-L. Yang, and M. Yosef. 2000. Maximum likelihood for hierarchical models via high-order multivariate Laplace approximations. Journal of Computational and Graphical Statistics 9: 141–157.
• Roberts, G. O. and S. K. Sahu. 1997. Updating schemes, correlation structure, blocking and parameterization for the Gibbs sampler. Journal of the Royal Statistical Society (Series B) 59: 291–318.
• Robinson, G. K. 1991. That BLUP is a good thing: the estimation of random effects (with discussion). Statistical Science 6: 15–51.
• Rodríguez, G. and N. Goldman. 1995. An assessment of estimation procedures for multilevel models with binary responses. Journal of the Royal Statistical Society (Series A) 158: 73–89.
• –-. 2001. Improved estimation procedures for multilevel models with binary responses: a case study. Journal of the Royal Statistical Society (Series A) 164: 339–355.
• Rubin, D. B. 1984. Bayesianly justifiable and relevant frequency calculations for the applied statistician. Annals of Statistics 12: 1151–1172.
• Samaniego, F. J. and D. M. Reneau. 1994. Toward a reconciliation of the Bayesian and frequentist approaches to point estimation. Journal of the American Statistical Association 89: 947–957.
• SAS-Institute. 2006. SAS Documentation, Release 9. Cary, NC: SAS Institute.
• Scheffé, H. 1959. The Analysis of Variance. New York: Wiley.
• Searle, S. R., G. Casella, and C. E. McCulloch. 1992. Variance Components. New York: Wiley.
• Seltzer, M. H. 1993. Sensitivity analysis for fixed effects in the hierarchical model: a Gibbs sampling approach. Journal of Educational Statistics 18: 207–235.
• Seltzer, M. H., W. H. Wong, and A. S. Bryk. 1996. Bayesian analysis in applications of hierarchical models: issues and methods. Journal of Educational and Behavioral Statistics 21: 131–167.
• Severini, T. A. 1994. On the relationship between Bayesian and non-Bayesian interval estimates. Journal of the Royal Statistical Society (Series B) 53: 611–618.
• Singh, A. C., D. M. Stukel, and D. Pfefferman. 1998. Bayesian versus frequentist measures of error in small area estimation. Journal of the Royal Statistical Society (Series B) 60: 377–396.
• Spiegelhalter, D. J., A. Thomas, N. Best, and W. R. Gilks. 1997. BUGS: Bayesian Inference Using Gibbs Sampling, Version 0.60. Cambridge: Medical Research Council Biostatistics Unit.
• Spiegelhalter, D. J., A. Thomas, N. Best, and D. Lunn. 2003. WinBUGS User Manual, Version 1.4. Available at www.mrc-bsu.cam.ac.uk/bugs.
• StataCorp. 2006. Stata Statistical Software: Release 9. College Station TX: Stata Corporation.
• Swallow, W. H. and J. F. Monahan. 1984. Monte Carlo comparison of ANOVA, MIVQUE, REML, and ML estimators of variance components. Technometrics 26: 47–57.
• White, H. 1980. A heteroscedasticity-consistent covariance matrix estimator and a direct test for heteroscedasticity. Econometrica 48: 817–830.
• Wilson, E. B. and M. M. Hilferty. 1931. The distribution of chi-square. Proceedings of the US National Academy of Sciences 17: 684.
• Woodhouse, G., J. Rasbash, H. Goldstein, M. Yang, J. Howarth, and I. Plewis. 1995. A Guide to MLn for New Users. London: Institute of Education, University of London.
• Zeger, S. L. and M. R. Karim. 1991. Generalized linear models with random effects: a Gibbs sampling approach. Journal of the American Statistical Association 86: 79–86.