The Annals of Statistics

Profile-kernel likelihood inference with diverging number of parameters

Clifford Lam and Jianqing Fan

Full-text: Open access

Enhanced PDF (368 KB)

Abstract

The generalized varying coefficient partially linear model with a growing number of predictors arises in many contemporary scientific endeavor. In this paper we set foot on both theoretical and practical sides of profile likelihood estimation and inference. When the number of parameters grows with sample size, the existence and asymptotic normality of the profile likelihood estimator are established under some regularity conditions. Profile likelihood ratio inference for the growing number of parameters is proposed and Wilk’s phenomenon is demonstrated. A new algorithm, called the accelerated profile-kernel algorithm, for computing profile-kernel estimator is proposed and investigated. Simulation studies show that the resulting estimates are as efficient as the fully iterative profile-kernel estimates. For moderate sample sizes, our proposed procedure saves much computational time over the fully iterative profile-kernel one and gives stabler estimates. A set of real data is analyzed using our proposed algorithm.

Article information

Source
Ann. Statist., Volume 36, Number 5 (2008), 2232-2260.

Dates
First available in Project Euclid: 13 October 2008

Permanent link to this document
https://projecteuclid.org/euclid.aos/1223908091

Digital Object Identifier
doi:10.1214/07-AOS544

Mathematical Reviews number (MathSciNet)
MR2458186

Zentralblatt MATH identifier
1274.62289

Subjects
Primary: 62G08: Nonparametric regression
Secondary: 62J12: Generalized linear models 62F12: Asymptotic properties of estimators

Keywords
Generalized linear models varying coefficients high dimensionality asymptotic normality profile likelihood generalized likelihood ratio tests

Citation

Lam, Clifford; Fan, Jianqing. Profile-kernel likelihood inference with diverging number of parameters. Ann. Statist. 36 (2008), no. 5, 2232--2260. doi:10.1214/07-AOS544. https://projecteuclid.org/euclid.aos/1223908091


Export citation

References

  • [1] Ahmad, I., Leelahanon, S. and Li, Q. (2005). Efficient estimation of a semiparametric partially linear varying coefficient model. Ann. Statist. 33 258–283.
    Mathematical Reviews (MathSciNet): MR2157803
    Zentralblatt MATH: 1064.62043
    Digital Object Identifier: doi:10.1214/009053604000000931
    Project Euclid: euclid.aos/1112967706
  • [2] Albright, S. C., Winston, W. L. and Zappe, C. J. (1999). Data Analysis and Decision Making \nwith Microsoft Excel. Pacific Grove, Duxbury, CA. Available at \nhttp://www.alibris.com/booksearch.detailinvid=9470354547&browse=1&qwork=1492588&qsort=&page=1.
  • [3] Bickel, P. J. (1975). One-step Huber estimates in linear models. J. Amer. Statist. Assoc. 70 428–433.
    Mathematical Reviews (MathSciNet): MR386168
    Zentralblatt MATH: 0322.62038
    Digital Object Identifier: doi:10.2307/2285834
    JSTOR: links.jstor.org
  • [4] Cai, Z., Fan, J. and Li, R. (2000). Efficient estimation and inferences for varying-coefficient models. J. Amer. Statist. Assoc. 95 888–902.
    Mathematical Reviews (MathSciNet): MR1804446
    Zentralblatt MATH: 0999.62052
    Digital Object Identifier: doi:10.2307/2669472
    JSTOR: links.jstor.org
  • [5] Donoho, D. L. (2000). High-dimensional data analysis: The curses and blessings of dimensionality. Lecture on August 8, 2000, to the American Mathematical Society on “Math Challenges of the 21st Century.” Available at http://www.inma.ucl.ac.be/~francois/these/papers/entry-Donoho-2000.html.
  • [6] Fan, J. and Huang, T. (2005). Profile likelihood inferences on semiparametric varying-coefficient partially linear models. Bernoulli 11 1031–1057.
    Mathematical Reviews (MathSciNet): MR2189080
    Digital Object Identifier: doi:10.3150/bj/1137421639
    Project Euclid: euclid.bj/1137421639
  • [7] Fan, J. and Li, R. (2006). Statistical challenges with high-dimensionality: Feature selection in knowledge discovery. Proceedings of International Congress of Mathematicians (M. Sanz-Solé, J. Soria, J. L. Varona and J. Verdera, eds.) III 595–622.
    Mathematical Reviews (MathSciNet): MR2275698
    Zentralblatt MATH: 1117.62137
  • [8] Fan, J. and Peng, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. Ann. Statist. 32 928–961.
    Mathematical Reviews (MathSciNet): MR2065194
    Zentralblatt MATH: 1092.62031
    Digital Object Identifier: doi:10.1214/009053604000000256
    Project Euclid: euclid.aos/1085408491
  • [9] Fan, J., Peng, H. and Huang, T. (2005). Semilinear high-dimensional model for normalization of microarray data: A theoretical analysis and partial consistency (with discussion). J. Amer. Statist. 100 781–813.
    Mathematical Reviews (MathSciNet): MR2201010
    Zentralblatt MATH: 1117.62330
    Digital Object Identifier: doi:10.1198/016214504000001781
  • [10] Fan, J., Zhang, C. and Zhang, J. (2001). Generalized likelihood ratio statistics and Wilks phenomenon. Ann. Statist. 29 153–193.
    Mathematical Reviews (MathSciNet): MR1833962
    Zentralblatt MATH: 1029.62042
    Digital Object Identifier: doi:10.1214/aos/996986505
    Project Euclid: euclid.aos/996986505
  • [11] Hastie, T. J. and Tibshirani, R. (1993). Varying-coefficient models. J. Roy. Statist. Soc. Ser. B 55 757–796.
    Mathematical Reviews (MathSciNet): MR1229881
    JSTOR: links.jstor.org
  • [12] Hu, Z., Wang, N. and Carroll, R. J. (2004). Profile-kernel versus backfitting in the partially linear models for longitudinal/clustered data. Biometrika 91 251–262.
    Mathematical Reviews (MathSciNet): MR2081299
    Zentralblatt MATH: 05001278
    Digital Object Identifier: doi:10.1093/biomet/91.2.251
  • [13] Huber, P. J. (1973). Robust regression: Asymptotics, conjectures and Monte Carlo. Ann. Statist. 1 799–821.
    Mathematical Reviews (MathSciNet): MR356373
    Zentralblatt MATH: 0289.62033
    Digital Object Identifier: doi:10.1214/aos/1176342503
    Project Euclid: euclid.aos/1176342503
  • [14] Jain, N. and Marcus, M. (1975). Central limit theorems for C(S)-valued random variables. J. Funct. Anal. 19 216–231.
    Mathematical Reviews (MathSciNet): MR385994
    Digital Object Identifier: doi:10.1016/0022-1236(75)90056-7
  • [15] Lam, C. and Fan, J. (2007). Profile-kernel likelihood inference with diverging number of parameters. Available at http://arxiv.org/PS_cache/math/pdf/0701/0701004v2.pdf.
    Mathematical Reviews (MathSciNet): MR2458186
    Zentralblatt MATH: 05368490
    Digital Object Identifier: doi:10.1214/07-AOS544
    Project Euclid: euclid.aos/1223908091
  • [16] Li, Q., Huang, C. J., Li., D. and Fu, T. T. (2002). Semiparametric smooth coefficient models. J. Bus. Econom. Statist. 20 412–422.
    Mathematical Reviews (MathSciNet): MR1939909
    JSTOR: links.jstor.org
  • [17] Li, R. and Liang, H. (2008). Variable selection in semiparametric regression modeling. Ann. Statist. 36 261–286.
    Mathematical Reviews (MathSciNet): MR2387971
    Zentralblatt MATH: 1132.62027
    Digital Object Identifier: doi:10.1214/009053607000000604
    Project Euclid: euclid.aos/1201877301
  • [18] Lin, X. and Carroll, R. J. (2006). Semiparametric estimation in general repeated measures problems. J. Roy. Statist. Soc. Ser. B 68 69–88.
    Mathematical Reviews (MathSciNet): MR2212575
    Zentralblatt MATH: 1141.62026
    Digital Object Identifier: doi:10.1111/j.1467-9868.2005.00533.x
  • [19] McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd ed. Chapman and Hall, London.
    Mathematical Reviews (MathSciNet): MR727836
  • [20] Murphy, S. A. (1993). Testing for a time dependent coefficient in Cox’s regression model. Scand. J. Statist. 20 35–50.
    Mathematical Reviews (MathSciNet): MR1221960
  • [21] Murphy, S. A. and van der Vaart, A. W. (2000). On profile likelihood (with discussion). J. Amer. Statist. Assoc. 95 449–485.
    Mathematical Reviews (MathSciNet): MR1803168
    Zentralblatt MATH: 0995.62033
    Digital Object Identifier: doi:10.2307/2669386
    JSTOR: links.jstor.org
  • [22] Portnoy, S. (1988). Asymptotic behavior of likelihood methods for exponential families when the number of parameters tends to infinity. Ann. Statist. 16 356–366.
    Mathematical Reviews (MathSciNet): MR924876
    Zentralblatt MATH: 0637.62026
    Digital Object Identifier: doi:10.1214/aos/1176350710
    Project Euclid: euclid.aos/1176350710
  • [23] Robinson, P. M. (1988). The stochastic difference between econometric and statistics. Econometrica 56 531–547.
    Mathematical Reviews (MathSciNet): MR946120
    Digital Object Identifier: doi:10.2307/1911699
    JSTOR: links.jstor.org
  • [24] Severini, T. A. and Wong, W. H. (1992). Profile likelihood and conditionally parametric models. Ann. Statist. 20 1768–1802.
    Mathematical Reviews (MathSciNet): MR1193312
    Zentralblatt MATH: 0768.62015
    Digital Object Identifier: doi:10.1214/aos/1176348889
    Project Euclid: euclid.aos/1176348889
  • [25] Van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Univ. Press.
    Mathematical Reviews (MathSciNet): MR1652247
    Zentralblatt MATH: 0910.62001
  • [26] Van Keilegom, I. and Carroll, R. J. (2007). Backfitting versus profiling in general criterion functions. Statist. Sinica 17 797–816.
    Mathematical Reviews (MathSciNet): MR2398434
    Zentralblatt MATH: 1128.62054
  • [27] Xia, Y., Zhang, W. and Tong, H. (2004). Efficient estimation for semivarying-coefficient models. Biometrika 91 661–681.
    Mathematical Reviews (MathSciNet): MR2090629
    Zentralblatt MATH: 1108.62019
    Digital Object Identifier: doi:10.1093/biomet/91.3.661
  • [28] Yatchew, A. (1997). An elementary estimator for the partially linear model. Economics Lett. 57 135–143.
    Mathematical Reviews (MathSciNet): MR1600233
    Zentralblatt MATH: 0896.90052
    Digital Object Identifier: doi:10.1016/S0165-1765(97)00218-8
  • [29] Zhang, W., Lee, S. Y. and Song, X. Y. (2002). Local polynomial fitting in semivarying coefficient model. J. Multivariate Anal. 82 166–188.
    Mathematical Reviews (MathSciNet): MR1918619
    Zentralblatt MATH: 0995.62038
    Digital Object Identifier: doi:10.1006/jmva.2001.2012

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more