Bernoulli

• Bernoulli
• Volume 6, Number 2 (2000), 243-284.

Likelihood-based inference with singular information matrix

Abstract

We consider likelihood-based asymptotic inference for a p-dimensional parameter θ of an identifiable parametric model with singular information matrix of rank p-1 at θ=θ* and likelihood differentiable up to a specific order. We derive the asymptotic distribution of the likelihood ratio test statistics for the simple null hypothesis that θ=θ* and of the maximum likelihood estimator (MLE) of θ when θ=θ*. We show that there exists a reparametrization such that the MLE of the last p-1 components of θ converges at rate$O_{p}\left( n^{-1/2}\right)$ . For the first component θ1 of θ the rate of convergence depends on the order s of the first non-zero partial derivative of the log-likelihood with respect to θ1 evaluated at θ*. When s is odd the rate of convergence of the MLE of θ1 is$O_{p}\left(n^{-1/2s}\right)$ . When s is even, the rate of convergence of the MLE of$\left|\theta_{1}-\theta_{1}^{\ast}\right|$ is$O_{p}\left(n^{-1/2s}\right)$ and, moreover, the asymptotic distribution of the sign of the MLE of θ11* is non-standard. When p=1 it is determined by the sign of the sum of the residuals from the population least-squares regression of the (s+1)th derivative of the individual contributions to the log-likelihood on their derivatives of order s. For p>1, it is determined by a linear combination of the sum of residuals of a multivariate population least-squares regression involving partial and mixed derivatives of the log-likelihood of a specific order. Thus although the MLE of 11*| has a uniform rate of convergence of$O_{p}\left(n^{-1/2s}\right)$ , the uniform convergence rate for the MLE of θ1 in suitable shrinking neighbourhoods of θ1* is only$O_{p}\left(n^{-1/\left(2s+2\right)}\right)$ .

Article information

Source
Bernoulli, Volume 6, Number 2 (2000), 243-284.

Dates
First available in Project Euclid: 12 April 2004

https://projecteuclid.org/euclid.bj/1081788028

Mathematical Reviews number (MathSciNet)
MR2001c:62020

Zentralblatt MATH identifier
0976.62015

Citation

Rotnitzky, Andrea; Cox, David R.; Bottai, Matteo; Robins, James. Likelihood-based inference with singular information matrix. Bernoulli 6 (2000), no. 2, 243--284. https://projecteuclid.org/euclid.bj/1081788028

References

• [1] Barndorff-Nielsen, O.E. and Cox, D.R. (1989) Asymptotic Techniques for Use in Statistics. London: Chapman & Hall.
• [2] Bhattacharyya, A. (1946) On some analogues to the amount of information and their uses in statistical estimation. Sankhya, 8, 1-14, 201-208, 277-280.
• [3] Bickel, P., Klaassen, C.A.J., Ritov, J. and Wellner, J.A. (1993) Efficient and Adaptive Inference in Semiparametric Models. Baltimore, MD: Johns Hopkins University Press.
• [4] Chant, D. (1974) On asymptotic tests of composite hypotheses in non-standard conditions. Biometrika, 61, 291-298.
• [5] Chen, J. (1995) Optimal rate of convergence for finite mixture models. Ann. Statist., 23, 221-233.
• [6] Chernoff, H. (1954) On the distribution of the likelihood ratio. Ann. Math. Statist., 25, 573-578.
• [7] Copas, J.B. and Li, H.G. (1997) Inference for non-random samples (with discussion). J. Roy. Statist. Soc. Ser. B, 59, 55-95.
• [8] Cox, D.R. and Hinkley, D.V. (1974) Theoretical Statistics. London: Chapman & Hall.
• [9] Cox, D.R. and Reid, N. (1987a) Approximations to noncentral distributions. Canad. J. Statist., 15, 105-114.
• [10] Cox, D.R. and Reid, N. (1987b) Parameter orthogonality and approximate conditional inference (with discussion). J. Roy. Statist. Soc. Ser. B, 49, 1-39.
• [11] Diggle, P. and Kenward, M.G. (1994) Informative drop-out in longitudinal data analysis (with discussion). Appl. Statist., 43, 49-93.
• [12] Faà di Bruno, F. (1859) Théorie Générale de l'Élimination. Paris: De Leiber et Faraquet.
• [13] Geyer, C. J. (1994) On the asymptotics of constrained M-estimation. Ann. Statist., 22, 1993- 2010.
• [14] Gronau, R. (1974) Wage comparisons: a selectivity bias. J. Political Economy, 82, 1119-1143.
• [15] Hájek, J. and Šidák, Z. (1967) Theory of Rank Tests. New York: Academic Press.
• [16] Heckman, J.J. (1976) The common structure of statistical models of truncation, sample selection and limited dependent variables, and a simple estimator for such models. Ann. Econom. Social Measurement, 5, 475-492.
• [17] Ibragimov, I.A. and Has'minskii, R.Z. (1981) Statistical Estimation, Asymptotic Theory. New York: Springer-Verlag.
• [18] Kiefer, N.M. (1982) A remark on the parameterization of a model for heterogeneity. Working paper no 278. Department of Economics, Cornell University, Ithaca, NY.
• [19] Lee, L.F. (1981) A specification test for normality in the generalized censored regression models. Discussion paper no 81-146. Center for Economic Research, University of Minnesota, Minneapolis.
• [20] Lee, L.F. (1993) Asymptotic distribution of the maximum likelihood estimator for a stochastic frontier function model with a singular information matrix. Econometric Theory, 9, 413-430.
• [21] Lee, L.F. and Chesher, A. (1986) Specification testing when score test statistic are identically zero. J. Econometrics, 31, 121-149.
• [22] Moran, P.A.P. (1971) Maximum likelihood estimators in non-standard conditions. Proc. Cambridge Philos. Soc., 70, 441-450.
• [23] Nelson, F.D. (1977) Censored regression models with unobserved stochastic censoring thresholds, J. Econometrics, 6, 309-327.
• [24] Newey, W. and McFadden, D. (1993) Estimation in large samples. In D. McFadden and R. Engler (eds), Handbook of Econometrics, Vol. 4. Amsterdam: North-Holland.
• [25] Rubin, D. B. (1976) Inference with missing data. Biometrika, 63, 581-592.
• [26] Rothenberg, T.J. (1971) Identification in parametric models. Econometrica, 39, 577-592.
• [27] Sargan, J.D. (1983) Identification and lack of identification. Econometrica, 51, 1605-1633.
• [28] Self, S. and Liang, K.Y. (1987) Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under non-standard conditions. J. Amer. Statist. Assoc., 82, 605-610.
• [29] Silvey, S.D. (1959) The Lagrangean multiplier test. Ann. Math. Statist., 30, 389-407.