• Bernoulli
  • Volume 17, Number 2 (2011), 736-748.

Semi-parametric regression: Efficiency gains from modeling the nonparametric part

Kyusang Yu, Enno Mammen, and Byeong U. Park

Full-text: Open access


It is widely admitted that structured nonparametric modeling that circumvents the curse of dimensionality is important in nonparametric estimation. In this paper we show that the same holds for semi-parametric estimation. We argue that estimation of the parametric component of a semi-parametric model can be improved essentially when more structure is put into the nonparametric part of the model. We illustrate this for the partially linear model, and investigate efficiency gains when the nonparametric part of the model has an additive structure. We present the semi-parametric Fisher information bound for estimating the parametric part of the partially linear additive model and provide semi-parametric efficient estimators for which we use a smooth backfitting technique to deal with the additive nonparametric part. We also present the finite sample performances of the proposed estimators and analyze Boston housing data as an illustration.

Article information

Bernoulli, Volume 17, Number 2 (2011), 736-748.

First available in Project Euclid: 5 April 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

partially linear additive models profile estimator semi-parametric efficiency smooth backfitting


Yu, Kyusang; Mammen, Enno; Park, Byeong U. Semi-parametric regression: Efficiency gains from modeling the nonparametric part. Bernoulli 17 (2011), no. 2, 736--748. doi:10.3150/10-BEJ296.

Export citation


  • [1] Bhattacharya, P. and Zhao, P. (1997). Semiparametric inference in a partial linear model. Ann. Statist. 25 244–262.
  • [2] Bickel, P. (1982). On adaptive estimation. Ann. Statist. 10 647–671.
  • [3] Bickel, P., Klaassen, A., Ritov, Y. and Wellner, J. (1993). Efficient and Adaptive Estimation for Semiparametric Models. Baltimore, MD: Johns Hopkins Univ. Press.
  • [4] Carroll, R., Maity, A., Mammen, E. and Yu, K. (2009). Efficient semiparametric marginal estimation for the partially linear additive model for longitudinal/clustered data. Statist. Biosci. 1 10–31.
  • [5] Cuzick, J. (1992). Efficient estimates in semiparametric additive regression models with unknown error distribution. Ann. Statist. 20 1129–1136.
  • [6] Fan, J. and Huang, T. (2005). Profile likelihood inferences on semiparametric varying-coefficient partially linear models. Bernoulli 11 1031–1057.
  • [7] Koul, H.L. and Schick, A. (1997). Efficient estimation in nonlinear autoregressive time-series models. Bernoulli 3 247–277.
  • [8] Liang, H., Thurston, S., Ruppert, D., Apanasovich, T. and Hauser, R. (2008). Additive partial linear models with measurement errors. Biometrika 95 667–678.
  • [9] Lu, Z., Lundervold, L., Tjøstheim, D. and Yao, Q. (2007). Exploring spatial nonlinearity using additive approximation. Bernoulli 13 447–472.
  • [10] Mammen, E., Linton, O. and Nielsen, J. (1999). The existence and asymptotic properties of a backfitting projection algorithm under weak conditions. Ann. Statist. 27 1443–1490.
  • [11] Mammen, E. and Park, B.U. (2005). Bandwidth selection for smooth backfitting in additive models. Ann. Statist. 33 1260–1294.
  • [12] Murphy, S. and van der Vaart, A. (2000). On profile likelihood (with comments). J. Amer. Statist. Assoc. 95 449–485.
  • [13] Opsomer, J. and Ruppert, D. (1999). A root-n consistent backfitting estimator for semiparametric additive modeling. J. Computat. Graph. Statist. 8 715–732.
  • [14] Park, B.U. (1990). Efficient estimation in the two sample semiparametric location-scale model. Probab. Theory Related Fields 86 21–39.
  • [15] Park, B.U. (1993). A cross-validatory choice of smoothing parameter in adaptive location estimation. J. Amer. Statist. Assoc. 88 848–854.
  • [16] Schick, A. (1986). On asymptotically efficient estimation in semiparametric models. Ann. Statist. 14 1139–1151.
  • [17] Schick, A. (1993). On efficient estimation in regression models. Ann. Statist. 21 1486–1521.
  • [18] Severini, T. and Wong, W. (1992). Profile likelihood and conditionally parametric models. Ann. Statist. 20 1768–1802.
  • [19] Speckman, P. (1988). Kernel smoothing in partial linear models. J. Roy. Statist. Soc. Ser. B 50 413–436.
  • [20] van de Geer, S. (2000) Empirical Processes in M-Estimation. Cambridge: Cambridge Univ. Press.
  • [21] van der Vaart, A. and Wellner, J. (1996) Weak Convergence and Empirical Processes. With Applications to Statistics. New York: Springer.
  • [22] Wang, J. and Yang, L. (2009). Efficient and fast spline-backfitted kernel smoothing of additive models. Ann. Inst. Statist. Math. 61 663–690.
  • [23] Yu, K., Mammen, E. and Park, B.U. (2008). Smooth backfitting in generalized additive models. Ann. Statist. 36 228–260.