The Annals of Statistics

Maximum likelihood methods for a generalized class of log-linear models

Joseph B. Lang

Full-text: Open access


We discuss maximum likelihood methods for fitting a broad class of multivariate categorical response data models. In particular, we derive the large-sample distributions for maximum likelihood estimators of parameters of product-multinomial generalized log-linear models. The large-sample behavior of other relevant likelihood-based statistics such as goodness-of-fit statistics and adjusted residuals is also described. The asymptotic results are derived within the framework of the constraint specification, rather than the more common freedom specification, of the model. We also outline an improved fitting algorithm for computing parameter maximum likelihood estimates and other relevant statistics. The broad class of multivariate categorical response data models, which are referred to as generalized log-linear models, can imply structure on several response configuration distributions (e.g., joint and marginal distributions). These models, which include as special cases log-linear, logit and cumulative-logit models, enjoy a wide breadth of application including longitudinal, rater-agreement and crossover data analyses.

Article information

Ann. Statist., Volume 24, Number 2 (1996), 726-752.

First available in Project Euclid: 24 September 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62H17: Contingency tables 62E20: Asymptotic distribution theory

Asymptotics constraint equation freedom equation marginal model multinomial distribution multivariate categorical data simultaneous model


Lang, Joseph B. Maximum likelihood methods for a generalized class of log-linear models. Ann. Statist. 24 (1996), no. 2, 726--752. doi:10.1214/aos/1032894462.

Export citation


  • AGRESTI, A. 1990. Categorical Data Analy sis. Wiley, New York. Z.
  • AITCHISON, J. 1962. Large-sample restricted parametric tests. J. Roy. Statist. Soc. Ser. B 24 234 250. Z.
  • AITCHISON, J. and SILVEY, S. D. 1958. Maximum-likelihood estimation of parameters subject to restraints. Ann. Math. Statist. 29 813 828. Z.
  • AITCHISON, J. and SILVEY, S. D. 1960. Maximum-likelihood estimation procedures and associated tests of significance. J. Roy. Statist. Soc. Ser. B 22 154 171. Z.
  • BECKER, M. P. and BALAGTAS, C. C. 1993. A log-nonlinear model for binary cross-over data. Biometrics 49 997 1009. Z.
  • BISHOP, Y., FIENBERG, S. E. and HOLLAND, P. 1975. Discrete Multivariate Analy sis. MIT Press. Z.
  • DALE, J. R. 1986. Global cross-ratio models for bivariate, discrete, ordered responses. Biometrics 42 909 917. Z.
  • GILULA, Z. and HABERMAN, S. J. 1986. Canonical analysis of contingency tables by maximum likelihood. J. Amer. Statist. Assoc. 81 780 798. Z.
  • GLONEK, G. F. V. and MCCULLAGH, P. 1995. Multivariate logistic models. J. Roy. Statist. Soc. Ser. B 57 533 546 Z.
  • HABER, M. 1985a. Maximum likelihood methods for linear and log-linear models in categorical data. Comput. Statist. Data Anal. 3 1 10. Z.
  • HABER, M. 1985b. Log-linear models for correlated marginal totals of a contingency table. Comm. Statist. Theory Methods 14 2845 2856. Z.
  • HABER, M. and BROWN, M. 1986. Maximum likelihood methods for log-linear models when expected frequencies are subject to linear constraints. J. Amer. Statist. Assoc. 81 477 482. Z.
  • HABERMAN, S. J. 1973. The analysis of residuals in cross-classification tables. Biometrics 29 205 220. Z.
  • HABERMAN, S. J. 1974. The Analy sis of Frequency Data. Univ. Chicago Press. Z.
  • LAIRD, N. M. 1991. Topics of likelihood-based methods for longitudinal data analysis. Statist. Sinica 1 33 50. Z.
  • LANG, J. B. 1996. On the comparison of multinomial and Poisson loglinear models. J. Roy. Statist. Soc. Ser. B 58 253 266. Z.
  • LANG, J. B. and AGRESTI, A. 1994. Simultaneously modeling joint and marginal distributions of multivariate categorical responses. J. Amer. Statist. Assoc. 89 625 632. Z.
  • MACRAE, E. C. 1974. Matrix derivatives with an application to an adaptive linear decision problem. Ann. Statist. 2 337 346. Z.
  • MCCULLAGH, P. and NELDER, J. A. 1989. Generalized Linear Models. Chapman and Hall, London. Z.
  • MOLENBERGHS, G. and LESAFFRE, E. 1994. Marginal modelling of multivariate categorical data. Unpublished manuscript. Z.
  • PALMGREN, J. 1981. The Fisher information matrix for log linear models arguing conditionally on observed explanatory variables. Biometrika 68 563 566. Z.
  • PIERCE, D. A. and SCHAFER, D. W. 1986. Residuals in generalized linear models. J. Amer. Statist. Assoc. 81 977 986. Z.
  • PRATT, J. W. 1981. Concavity of the log likelihood. J. Amer. Statist. Assoc. 76 103 106. Z.
  • SERFLING, R. J. 1980. Approximation Theorems of Mathematical Statistics. Wiley, New York. Z.
  • SILVEY, S. D. 1959. The Lagrange-multiplier test. Ann. Math. Statist. 30 389 407.
  • IOWA CITY, IOWA 52242