• Bernoulli
  • Volume 6, Number 2 (2000), 243-284.

Likelihood-based inference with singular information matrix

Andrea Rotnitzky, David R. Cox, Matteo Bottai, and James Robins

Full-text: Open access


We consider likelihood-based asymptotic inference for a p-dimensional parameter θ of an identifiable parametric model with singular information matrix of rank p-1 at θ=θ* and likelihood differentiable up to a specific order. We derive the asymptotic distribution of the likelihood ratio test statistics for the simple null hypothesis that θ=θ* and of the maximum likelihood estimator (MLE) of θ when θ=θ*. We show that there exists a reparametrization such that the MLE of the last p-1 components of θ converges at rate O p ( n - 1/2 ) . For the first component θ1 of θ the rate of convergence depends on the order s of the first non-zero partial derivative of the log-likelihood with respect to θ1 evaluated at θ*. When s is odd the rate of convergence of the MLE of θ1 is O p ( n - 1/2s ) . When s is even, the rate of convergence of the MLE of | θ 1 -θ 1 * | is O p ( n - 1/2s ) and, moreover, the asymptotic distribution of the sign of the MLE of θ11* is non-standard. When p=1 it is determined by the sign of the sum of the residuals from the population least-squares regression of the (s+1)th derivative of the individual contributions to the log-likelihood on their derivatives of order s. For p>1, it is determined by a linear combination of the sum of residuals of a multivariate population least-squares regression involving partial and mixed derivatives of the log-likelihood of a specific order. Thus although the MLE of 11*| has a uniform rate of convergence of O p ( n - 1/2s ) , the uniform convergence rate for the MLE of θ1 in suitable shrinking neighbourhoods of θ1* is only O p ( n - 1/( 2s+2 ) ) .

Article information

Bernoulli, Volume 6, Number 2 (2000), 243-284.

First available in Project Euclid: 12 April 2004

Permanent link to this document

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

constraint estimation identifiability likelihood ratio test non-ignorable non-response reparametrization rate of convergence


Rotnitzky, Andrea; Cox, David R.; Bottai, Matteo; Robins, James. Likelihood-based inference with singular information matrix. Bernoulli 6 (2000), no. 2, 243--284.

Export citation


  • [1] Barndorff-Nielsen, O.E. and Cox, D.R. (1989) Asymptotic Techniques for Use in Statistics. London: Chapman & Hall.
  • [2] Bhattacharyya, A. (1946) On some analogues to the amount of information and their uses in statistical estimation. Sankhya, 8, 1-14, 201-208, 277-280.
  • [3] Bickel, P., Klaassen, C.A.J., Ritov, J. and Wellner, J.A. (1993) Efficient and Adaptive Inference in Semiparametric Models. Baltimore, MD: Johns Hopkins University Press.
  • [4] Chant, D. (1974) On asymptotic tests of composite hypotheses in non-standard conditions. Biometrika, 61, 291-298.
  • [5] Chen, J. (1995) Optimal rate of convergence for finite mixture models. Ann. Statist., 23, 221-233.
  • [6] Chernoff, H. (1954) On the distribution of the likelihood ratio. Ann. Math. Statist., 25, 573-578.
  • [7] Copas, J.B. and Li, H.G. (1997) Inference for non-random samples (with discussion). J. Roy. Statist. Soc. Ser. B, 59, 55-95.
  • [8] Cox, D.R. and Hinkley, D.V. (1974) Theoretical Statistics. London: Chapman & Hall.
  • [9] Cox, D.R. and Reid, N. (1987a) Approximations to noncentral distributions. Canad. J. Statist., 15, 105-114.
  • [10] Cox, D.R. and Reid, N. (1987b) Parameter orthogonality and approximate conditional inference (with discussion). J. Roy. Statist. Soc. Ser. B, 49, 1-39.
  • [11] Diggle, P. and Kenward, M.G. (1994) Informative drop-out in longitudinal data analysis (with discussion). Appl. Statist., 43, 49-93.
  • [12] Faà di Bruno, F. (1859) Théorie Générale de l'Élimination. Paris: De Leiber et Faraquet.
  • [13] Geyer, C. J. (1994) On the asymptotics of constrained M-estimation. Ann. Statist., 22, 1993- 2010.
  • [14] Gronau, R. (1974) Wage comparisons: a selectivity bias. J. Political Economy, 82, 1119-1143.
  • [15] Hájek, J. and Šidák, Z. (1967) Theory of Rank Tests. New York: Academic Press.
  • [16] Heckman, J.J. (1976) The common structure of statistical models of truncation, sample selection and limited dependent variables, and a simple estimator for such models. Ann. Econom. Social Measurement, 5, 475-492.
  • [17] Ibragimov, I.A. and Has'minskii, R.Z. (1981) Statistical Estimation, Asymptotic Theory. New York: Springer-Verlag.
  • [18] Kiefer, N.M. (1982) A remark on the parameterization of a model for heterogeneity. Working paper no 278. Department of Economics, Cornell University, Ithaca, NY.
  • [19] Lee, L.F. (1981) A specification test for normality in the generalized censored regression models. Discussion paper no 81-146. Center for Economic Research, University of Minnesota, Minneapolis.
  • [20] Lee, L.F. (1993) Asymptotic distribution of the maximum likelihood estimator for a stochastic frontier function model with a singular information matrix. Econometric Theory, 9, 413-430.
  • [21] Lee, L.F. and Chesher, A. (1986) Specification testing when score test statistic are identically zero. J. Econometrics, 31, 121-149.
  • [22] Moran, P.A.P. (1971) Maximum likelihood estimators in non-standard conditions. Proc. Cambridge Philos. Soc., 70, 441-450.
  • [23] Nelson, F.D. (1977) Censored regression models with unobserved stochastic censoring thresholds, J. Econometrics, 6, 309-327.
  • [24] Newey, W. and McFadden, D. (1993) Estimation in large samples. In D. McFadden and R. Engler (eds), Handbook of Econometrics, Vol. 4. Amsterdam: North-Holland.
  • [25] Rubin, D. B. (1976) Inference with missing data. Biometrika, 63, 581-592.
  • [26] Rothenberg, T.J. (1971) Identification in parametric models. Econometrica, 39, 577-592.
  • [27] Sargan, J.D. (1983) Identification and lack of identification. Econometrica, 51, 1605-1633.
  • [28] Self, S. and Liang, K.Y. (1987) Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under non-standard conditions. J. Amer. Statist. Assoc., 82, 605-610.
  • [29] Silvey, S.D. (1959) The Lagrangean multiplier test. Ann. Math. Statist., 30, 389-407.