The Annals of Statistics

Penalized maximum likelihood and semiparametric second-order efficiency

A. S. Dalalyan, G. K. Golubev, and A. B. Tsybakov

Full-text: Open access


We consider the problem of estimation of a shift parameter of an unknown symmetric function in Gaussian white noise. We introduce a notion of semiparametric second-order efficiency and propose estimators that are semiparametrically efficient and second-order efficient in our model. These estimators are of a penalized maximum likelihood type with an appropriately chosen penalty. We argue that second-order efficiency is crucial in semiparametric problems since only the second-order terms in asymptotic expansion for the risk account for the behavior of the “nonparametric component” of a semiparametric procedure, and they are not dramatically smaller than the first-order terms.

Article information

Ann. Statist., Volume 34, Number 1 (2006), 169-201.

First available in Project Euclid: 2 May 2006

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation 62G20: Asymptotic properties

Semiparametric estimation estimating a shift of a nonparametric function second-order efficiency penalized maximum likelihood exact minimax asymptotics


Dalalyan, A. S.; Golubev, G. K.; Tsybakov, A. B. Penalized maximum likelihood and semiparametric second-order efficiency. Ann. Statist. 34 (2006), no. 1, 169--201. doi:10.1214/009053605000000895.

Export citation


  • Belitser, E. N. and Levit, B. Ya. (1995). On minimax filtering over ellipsoids. Math. Methods Statist. 4 259–273.
  • Bickel, P. J. (1982). On adaptive estimation. Ann. Statist. 10 647–671.
  • Bickel, P. J., Klaassen, C. A. J., Ritov, Y. and Wellner, J. A. (1998). Efficient and Adaptive Estimation for Semiparametric Models. Springer, New York.
  • Cavalier, L., Golubev, G. K., Picard, D. and Tsybakov, A. B. (2002). Oracle inequalities for inverse problems. Ann. Statist. 30 843–874.
  • Cramér, H. and Leadbetter, M. R. (1967). Stationary and Related Stochastic Processes. Wiley, New York.
  • Dalalyan, A. S. and Kutoyants, Yu. A. (2004). On second order minimax estimation of invariant density for ergodic diffusion. Statist. Decisions 22 17–41.
  • Gill, R. D. and Levit, B. Ya. (1995). Applications of the Van Trees inequality: A Bayesian Cramér–Rao bound. Bernoulli 1 59–79.
  • Golubev, G. K. (1990). On estimation of the time delay of a signal under nuisance parameters. Problems Inform. Transmission 25 173–180.
  • Golubev, G. and Härdle, W. (2000). Second order minimax estimation in partial linear models. Math. Methods Statist. 9 160–175.
  • Golubev, G. and Härdle, W. (2002). On adaptive smoothing in partial linear models. Math. Methods Statist. 11 98–117.
  • Golubev, G. K. and Levit, B. Ya. (1996). On the second order minimax estimation of distribution functions. Math. Methods Statist. 5 1–31.
  • Golubev, G. K. and Levit, B. Ya. (1996). Asymptotically efficient estimation for analytic distributions. Math. Methods Statist. 5 357–368.
  • Hallin, M. and Werker, B. J. M. (2003). Semi-parametric efficiency, distribution-freeness and invariance. Bernoulli 9 137–165.
  • Härdle, W. and Marron, J. S. (1990). Semiparametric comparison of regression curves. Ann. Statist. 18 63–89.
  • Härdle, W. and Tsybakov, A. B. (1993). How sensitive are average derivatives? J. Econometrics 58 31–48.
  • Hida, T. (1980) Brownian Motion. Springer, New York.
  • Ibragimov, I. A. and Hasminskii, R. Z. (1981). Statistical Estimation. Asymptotic Theory. Springer, New York.
  • Leadbetter, M. R., Lindgren, G. and Rootzén, H. (1986). Extremes and Related Properties of Random Sequences and Processes. Springer, New York.
  • Le Cam, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York.
  • Levit, B. Ya. (1985). Second-order asymptotic optimality and positive solutions of the Schrödinger equation. Theory Probab. Appl. 30 333–363.
  • Mammen, E. and Park, B. (1997). Optimal smoothing in adaptive location estimation. J. Statist. Plann. Inference 58 333–348.
  • Pfanzagl, J. (1990). Estimation in Semiparametric Models. Lecture Notes in Statist. 63. Springer, New York.
  • Pinsker, M. S. (1980). Optimal filtering of square integrable signals in Gaussian white noise. Problems Inform. Transmission 16 120–133.
  • Schick, A. (1986). On asymptotically efficient estimation in semiparametric models. Ann. Statist. 14 1139–1151.
  • Severini, T. A. and Wong, W. H. (1992). Profile likelihood and conditionally parametric models. Ann. Statist. 20 1768–1802.
  • Stein, C. (1956). Efficient nonparametric testing and estimation. Proc. Third Berkeley Symp. Math. Statist. Probab. 1 187–195. Univ. California Press, Berkeley.
  • Stone, C. J. (1975). Adaptive maximum likelihood estimators of a location parameter. Ann. Statist. 3 267–284.
  • Tsybakov, A. B. (1998). Pointwise and sup-norm sharp adaptive estimation of functions on the Sobolev classes. Ann. Statist. 26 2420–2469.
  • Tsybakov, A. B. (2004). Introduction à l'estimation non-paramétrique. Springer, Berlin.
  • van der Laan, M. J. (1995). Efficient and Inefficient Estimation in Semiparametric Models. CWI, Amsterdam.
  • van der Vaart, A. (1996). Efficient maximum likelihood estimation in semiparametric mixture models. Ann. Statist. 24 862–878.
  • van der Vaart, A. (1998). Asymptotic Statistics. Cambridge Univ. Press.
  • Van Trees, H. L. (1968). Detection, Estimation and Modulation Theory 1. Wiley, New York.