Annals of Statistics

Profile-kernel likelihood inference with diverging number of parameters

Clifford Lam and Jianqing Fan

Full-text: Open access


The generalized varying coefficient partially linear model with a growing number of predictors arises in many contemporary scientific endeavor. In this paper we set foot on both theoretical and practical sides of profile likelihood estimation and inference. When the number of parameters grows with sample size, the existence and asymptotic normality of the profile likelihood estimator are established under some regularity conditions. Profile likelihood ratio inference for the growing number of parameters is proposed and Wilk’s phenomenon is demonstrated. A new algorithm, called the accelerated profile-kernel algorithm, for computing profile-kernel estimator is proposed and investigated. Simulation studies show that the resulting estimates are as efficient as the fully iterative profile-kernel estimates. For moderate sample sizes, our proposed procedure saves much computational time over the fully iterative profile-kernel one and gives stabler estimates. A set of real data is analyzed using our proposed algorithm.

Article information

Ann. Statist., Volume 36, Number 5 (2008), 2232-2260.

First available in Project Euclid: 13 October 2008

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression
Secondary: 62J12: Generalized linear models 62F12: Asymptotic properties of estimators

Generalized linear models varying coefficients high dimensionality asymptotic normality profile likelihood generalized likelihood ratio tests


Lam, Clifford; Fan, Jianqing. Profile-kernel likelihood inference with diverging number of parameters. Ann. Statist. 36 (2008), no. 5, 2232--2260. doi:10.1214/07-AOS544.

Export citation


  • [1] Ahmad, I., Leelahanon, S. and Li, Q. (2005). Efficient estimation of a semiparametric partially linear varying coefficient model. Ann. Statist. 33 258–283.
  • [2] Albright, S. C., Winston, W. L. and Zappe, C. J. (1999). Data Analysis and Decision Making \nwith Microsoft Excel. Pacific Grove, Duxbury, CA. Available at \n
  • [3] Bickel, P. J. (1975). One-step Huber estimates in linear models. J. Amer. Statist. Assoc. 70 428–433.
  • [4] Cai, Z., Fan, J. and Li, R. (2000). Efficient estimation and inferences for varying-coefficient models. J. Amer. Statist. Assoc. 95 888–902.
  • [5] Donoho, D. L. (2000). High-dimensional data analysis: The curses and blessings of dimensionality. Lecture on August 8, 2000, to the American Mathematical Society on “Math Challenges of the 21st Century.” Available at
  • [6] Fan, J. and Huang, T. (2005). Profile likelihood inferences on semiparametric varying-coefficient partially linear models. Bernoulli 11 1031–1057.
  • [7] Fan, J. and Li, R. (2006). Statistical challenges with high-dimensionality: Feature selection in knowledge discovery. Proceedings of International Congress of Mathematicians (M. Sanz-Solé, J. Soria, J. L. Varona and J. Verdera, eds.) III 595–622.
  • [8] Fan, J. and Peng, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. Ann. Statist. 32 928–961.
  • [9] Fan, J., Peng, H. and Huang, T. (2005). Semilinear high-dimensional model for normalization of microarray data: A theoretical analysis and partial consistency (with discussion). J. Amer. Statist. 100 781–813.
  • [10] Fan, J., Zhang, C. and Zhang, J. (2001). Generalized likelihood ratio statistics and Wilks phenomenon. Ann. Statist. 29 153–193.
  • [11] Hastie, T. J. and Tibshirani, R. (1993). Varying-coefficient models. J. Roy. Statist. Soc. Ser. B 55 757–796.
  • [12] Hu, Z., Wang, N. and Carroll, R. J. (2004). Profile-kernel versus backfitting in the partially linear models for longitudinal/clustered data. Biometrika 91 251–262.
  • [13] Huber, P. J. (1973). Robust regression: Asymptotics, conjectures and Monte Carlo. Ann. Statist. 1 799–821.
  • [14] Jain, N. and Marcus, M. (1975). Central limit theorems for C(S)-valued random variables. J. Funct. Anal. 19 216–231.
  • [15] Lam, C. and Fan, J. (2007). Profile-kernel likelihood inference with diverging number of parameters. Available at
  • [16] Li, Q., Huang, C. J., Li., D. and Fu, T. T. (2002). Semiparametric smooth coefficient models. J. Bus. Econom. Statist. 20 412–422.
  • [17] Li, R. and Liang, H. (2008). Variable selection in semiparametric regression modeling. Ann. Statist. 36 261–286.
  • [18] Lin, X. and Carroll, R. J. (2006). Semiparametric estimation in general repeated measures problems. J. Roy. Statist. Soc. Ser. B 68 69–88.
  • [19] McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd ed. Chapman and Hall, London.
  • [20] Murphy, S. A. (1993). Testing for a time dependent coefficient in Cox’s regression model. Scand. J. Statist. 20 35–50.
  • [21] Murphy, S. A. and van der Vaart, A. W. (2000). On profile likelihood (with discussion). J. Amer. Statist. Assoc. 95 449–485.
  • [22] Portnoy, S. (1988). Asymptotic behavior of likelihood methods for exponential families when the number of parameters tends to infinity. Ann. Statist. 16 356–366.
  • [23] Robinson, P. M. (1988). The stochastic difference between econometric and statistics. Econometrica 56 531–547.
  • [24] Severini, T. A. and Wong, W. H. (1992). Profile likelihood and conditionally parametric models. Ann. Statist. 20 1768–1802.
  • [25] Van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Univ. Press.
  • [26] Van Keilegom, I. and Carroll, R. J. (2007). Backfitting versus profiling in general criterion functions. Statist. Sinica 17 797–816.
  • [27] Xia, Y., Zhang, W. and Tong, H. (2004). Efficient estimation for semivarying-coefficient models. Biometrika 91 661–681.
  • [28] Yatchew, A. (1997). An elementary estimator for the partially linear model. Economics Lett. 57 135–143.
  • [29] Zhang, W., Lee, S. Y. and Song, X. Y. (2002). Local polynomial fitting in semivarying coefficient model. J. Multivariate Anal. 82 166–188.