The Annals of Statistics

Adaptive function estimation in nonparametric regression with one-sided errors

Moritz Jirak, Alexander Meister, and Markus Reiß

Full-text: Open access


We consider the model of nonregular nonparametric regression where smoothness constraints are imposed on the regression function $f$ and the regression errors are assumed to decay with some sharpness level at their endpoints. The aim of this paper is to construct an adaptive estimator for the regression function $f$. In contrast to the standard model where local averaging is fruitful, the nonregular conditions require a substantial different treatment based on local extreme values. We study this model under the realistic setting in which both the smoothness degree $\beta>0$ and the sharpness degree $\mathfrak{a}\in(0,\infty)$ are unknown in advance. We construct adaptation procedures applying a nested version of Lepski’s method and the negative Hill estimator which show no loss in the convergence rates with respect to the general $L_{q}$-risk and a logarithmic loss with respect to the pointwise risk. Optimality of these rates is proved for $\mathfrak{a}\in(0,\infty)$. Some numerical simulations and an application to real data are provided.

Article information

Ann. Statist., Volume 42, Number 5 (2014), 1970-2002.

First available in Project Euclid: 11 September 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression 62G32: Statistics of extreme values; tail inference

Adaptive convergence rates bandwidth selection frontier estimation Lepski’s method minimax optimality negative Hill estimator nonregular regression


Jirak, Moritz; Meister, Alexander; Reiß, Markus. Adaptive function estimation in nonparametric regression with one-sided errors. Ann. Statist. 42 (2014), no. 5, 1970--2002. doi:10.1214/14-AOS1248.

Export citation


  • [1] Aigner, D., Lovell, C. A. K. and Schmidt, P. (1977). Formulation and estimation of stochastic frontier production function models. J. Econometrics 6 21–37.
  • [2] Banker, R. D. (1993). Maximum likelihood, consistency and data envelopment analysis: A statistical foundation. Manage. Sci. 39 1265–1273.
  • [3] Beirlant, J., Goegebeur, Y., Teugels, J. and Segers, J. (2004). Statistics of Extremes: Theory and Applications. Wiley, Chichester.
  • [4] Chernozhukov, V. and Hong, H. (2004). Likelihood estimation and inference in a class of nonregular econometric models. Econometrica 72 1445–1480.
  • [5] Chichignoud, M. (2012). Minimax and minimax adaptive estimation in multiplicative regression: Locally Bayesian approach. Probab. Theory Related Fields 153 543–586.
  • [6] Daouia, A., Florens, J.-P. and Simar, L. (2010). Frontier estimation and extreme value theory. Bernoulli 16 1039–1063.
  • [7] de Haan, L. and Ferreira, A. (2006). Extreme Value Theory: An introduction. Springer Series in Operations Research and Financial Engineering. Springer, New York.
  • [8] de Haan, L. and Peng, L. (1998). Comparison of tail index estimators. Stat. Neerl. 52 60–70.
  • [9] Deprins, D., Simar, L. and Tulkens, H. (1984). Measuring labor-efficiency in post offices. In Public Goods, Environmental Externalities and Fiscal Competition (P. Chander, J. Drèze, C. K. Lovell and J. Mintz, eds.) 285–309. Springer, New York.
  • [10] Donald, S. G. and Paarsch, H. J. (1993). Piecewise pseudo-maximum likelihood estimation in empirical models of auctions. Internat. Econom. Rev. 34 121–148.
  • [11] Drees, H. (1995). Refined Pickands estimators of the extreme value index. Ann. Statist. 23 2059–2080.
  • [12] Drees, H. (1998). Optimal rates of convergence for estimates of the extreme value index. Ann. Statist. 26 434–448.
  • [13] Drees, H. (2001). Minimax risk bounds in extreme value theory. Ann. Statist. 29 266–294.
  • [14] Drees, H., de Haan, L. and Resnick, S. (2000). How to make a Hill plot. Ann. Statist. 28 254–274.
  • [15] Falk, M. (1995). Some best parameter estimates for distributions with finite endpoint. Statistics 27 115–125.
  • [16] Falk, M., Hüsler, J. and Reiss, R.-D. (1994). Laws of Small Numbers: Extremes and Rare Events. Birkhäuser, Basel.
  • [17] Farrell, M. J. (1957). The measurement of productive efficiency. J. R. Stat. Soc. A, General 120 253–290.
  • [18] Fraga Alves, M. I. (2001). A location invariant Hill-type estimator. Extremes 4 199–217 (2002).
  • [19] Gijbels, I., Mammen, E., Park, B. U. and Simar, L. (1999). On estimation of monotone and concave frontier functions. J. Amer. Statist. Assoc. 94 220–228.
  • [20] Gijbels, I. and Peng, L. (2000). Estimation of a support curve via order statistics. Extremes 3 251–277 (2001).
  • [21] Girard, S., Guillou, A. and Stupfler, G. (2013). Frontier estimation with kernel regression on high order moments. J. Multivariate Anal. 116 172–189.
  • [22] Goldenshluger, A. and Lepski, O. (2011). Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality. Ann. Statist. 39 1608–1632.
  • [23] Goldenshluger, A. and Zeevi, A. (2006). Recovering convex boundaries from blurred and noisy observations. Ann. Statist. 34 1375–1394.
  • [24] Grama, I. and Spokoiny, V. (2008). Statistics of extremes by oracle estimation. Ann. Statist. 36 1619–1648.
  • [25] Guerre, E., Perrigne, I. and Vuong, Q. (2000). Optimal nonparametric estimation of first-price auctions. Econometrica 68 525–574.
  • [26] Hall, P., Nussbaum, M. and Stern, S. E. (1997). On the estimation of a support curve of indeterminate sharpness. J. Multivariate Anal. 62 204–232.
  • [27] Hall, P. and Park, B. U. (2004). Bandwidth choice for local polynomial estimation of smooth boundaries. J. Multivariate Anal. 91 240–261.
  • [28] Hall, P., Park, B. U. and Stern, S. E. (1998). On polynomial estimators of frontiers and boundaries. J. Multivariate Anal. 66 71–98.
  • [29] Hall, P. and Van Keilegom, I. (2009). Nonparametric “regression” when errors are positioned at end-points. Bernoulli 15 614–633.
  • [30] Härdle, W., Park, B. U. and Tsybakov, A. B. (1995). Estimation of nonsharp support boundaries. J. Multivariate Anal. 55 205–218.
  • [31] He, L. (2001). Modeling and prediction of sunspot cycles. ProQuest LLC, Ann Arbor, MI. Ph.D. thesis, Massachusetts Institute of Technology. Available at
  • [32] Hosking, J. R. M. and Wallis, J. R. (1987). Parameter and quantile estimation for the generalized Pareto distribution. Technometrics 29 339–349.
  • [33] Ibragimov, I. A. and Hasminskiĭ, R. Z. (1981). Statistical Estimation: Asymptotic Theory. Springer, New York. Translated from the Russian by Samuel Kotz.
  • [34] Jeong, S.-O. and Park, B. U. (2006). Large sample approximation of the distribution for convex-hull estimators of boundaries. Scand. J. Stat. 33 139–151.
  • [35] Jirak, M., Meister, A. and Reiß, M. (2014). Supplement to “Adaptive function estimation in nonparametric regression with one-sided errors.” DOI:10.1214/14-AOS1248SUPP.
  • [36] Jirak, M. and Meister, A. Available at
  • [37] Kneip, A., Park, B. U. and Simar, L. (1998). A note on the convergence of nonparametric DEA estimators for production efficiency scores. Econometric Theory 14 783–793.
  • [38] Kneip, A., Simar, L. and Wilson, P. W. (2008). Asymptotics and consistent bootstraps for DEA estimators in nonparametric frontier models. Econometric Theory 24 1663–1697.
  • [39] Kneip, L., Simar, L. and Van Keilegom, I. (2010). Boundary estimation in the presence of measurement error with unknown variance. Unpublished manuscript.
  • [40] Knight, K. (2001). Limiting distributions of linear programming estimators. Extremes 4 87–103 (2002).
  • [41] Knight, K. (2006). Asymptotic theory for $M$-estimators of boundaries. In The Art of Semiparametrics (S. Sperlich, W. Härdle and G. Aydinli, eds.) 1–21. Physica-Verlag, Heidelberg.
  • [42] Koenker, R., Ng, P. and Portnoy, S. (1994). Quantile smoothing splines. Biometrika 81 673–680.
  • [43] Korostelëv, A. P., Simar, L. and Tsybakov, A. B. (1995). Efficient estimation of monotone boundaries. Ann. Statist. 23 476–489.
  • [44] Korostelëv, A. P. and Tsybakov, A. B. (1993). Minimax Theory of Image Reconstruction. Lecture Notes in Statistics 82. Springer, New York.
  • [45] Kumbhakar, S. C., Park, B. U., Simar, L. and Tsionas, E. G. (2007). Nonparametric stochastic frontiers: A local maximum likelihood approach. J. Econometrics 137 1–27.
  • [46] Lasne, F. and de Ceaurriz, J. (2000). Recombinant erythropoietin in urine. Nature 405 1084–1087.
  • [47] Lepski, O. V. and Spokoiny, V. G. (1997). Optimal pointwise adaptive methods in nonparametric estimation. Ann. Statist. 25 2512–2546.
  • [48] Lepskiĭ, O. V. (1990). A problem of adaptive estimation in Gaussian white noise. Teor. Veroyatn. Primen. 35 459–470.
  • [49] Meeusen, W. and van D. Broeck, J. (1977). Efficiency estimation from Cobb-Douglas production functions with composed error. Internat. Econom. Rev. 18 435–444.
  • [50] Meister, A. and Reiß, M. (2013). Asymptotic equivalence for nonparametric regression with nonregular errors. Probab. Theory Related Fields 155 201–229.
  • [51] Müller, U. U. and W., Wefelmeyer (2010). Estimation in nonparametric regression with nonregular errors. Comm. Statist. Theory Methods 39 1619–1629.
  • [52] Park, B. U., Jeong, S.-O. and Simar, L. (2010). Asymptotic distribution of conical-hull estimators of directional edges. Ann. Statist. 38 1320–1340.
  • [53] Park, B. U., Sickles, R. C. and Simar, L. (2007). Semiparametric efficient estimation of dynamic panel data models. J. Econometrics 136 281–301.
  • [54] Park, B. U., Simar, L. and Weiner, Ch. (2000). The FDH estimator for productivity efficiency scores. Econometric Theory 16 855–877.
  • [55] Pickands, J. III (1975). Statistical inference using extreme order statistics. Ann. Statist. 3 119–131.
  • [56] Solanki, S. K., Usoskin, I. G., Kromer, B., Schüssler, M. and Beer, J. (2004). Unusual activity of the sun during recent decades compared to the previous 11,000 years. Nature 431 1084–1087.

Supplemental materials

  • Supplementary material: Additional simulations, proof of lower bound, technical lemmas and sharpness estimation. In the supplementary material we provide additional simulations and the proofs of the lower bound results as well as technical lemmas and sharpness estimation.