Electronic Journal of Statistics

Finite sample behavior of a sieve profile estimator in the single index model

Andreas Andresen

Full-text: Open access


We apply the results of Andresen et. al. (2014) on finite sample properties of sieve M-estimators and Andresen et. al. (2015) on the convergence of an alternating maximization procedure to analyse a sieve profile maximization estimator in the single index model with linear index function. The link function is approximated with $C^{3}$-Daubechies-wavelets with compact support. We derive results like Wilks phenomenon and Fisher Theorem in a finite sample setup even when the model is miss-specified. Furthermore we show that an alternating maximization procedure converges to the global maximizer and we assess the performance of Friedman’s projection pursuit procedure. The approach is based on showing that the conditions of Andresen et. al. (2014) and (2015) can be satisfied under a set of mild regularity and moment conditions on the link function, the regressors and the additive noise. The results allow to construct non-asymptotic confidence sets and to derive asymptotic bounds for the estimator as corollaries.

Article information

Electron. J. Statist., Volume 9, Number 2 (2015), 2528-2641.

Received: April 2015
First available in Project Euclid: 19 November 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F10: Point estimation
Secondary: 62J12: Generalized linear models 62F25: Tolerance and confidence regions 62H12: Estimation

Profile estimator sieve projection pursuit procedure alternating maximization alternating minimization single index


Andresen, Andreas. Finite sample behavior of a sieve profile estimator in the single index model. Electron. J. Statist. 9 (2015), no. 2, 2528--2641. doi:10.1214/15-EJS1079. https://projecteuclid.org/euclid.ejs/1447943705

Export citation


  • [1] A. Andresen. A note on the bias of sieve profile estimation., arXiv :1406.4045, 2014.
  • [2] A. Andresen and V. Spokoiny. Critical dimension in profile semiparametric estimation., Electron. J. Statist., 8(2) :3077–3125, 2014.
  • [3] A. Andresen and V. Spokoiny. Two convergence results for an alternation maximization procedure., arXiv :1501.01525v1, 2014.
  • [4] A. Cohen, I. Daubechies, and P. Vial. Wavelets on the interval and fast wavelet transforms., Applied and computational harmonic analysis, 1:54–81, 1993.
  • [5] M. Delecroix, W. Haerdle, and M. Hristache. Efficient estimation in single-index regression. Technical report, SFB 373, Humboldt Univ. Berlin, 1997.
  • [6] R. M. Dudley. The sizes of compact subsets of hilbert space and continuity of gaussian processes., Journal of Functional Analysis, 1:290–330, 1967.
  • [7] Jerome H. Friedman and Werner Stuetzle. Projection pursuit regression., Journal of the American Statistical Association, 76(376):817–823, 1981.
  • [8] W. Haerdle, P. Hall, and H. Ichimura. Optimal smoothing in single-index models., Ann. Statist., 21:157–178, 1993.
  • [9] Peter Hall. Estimating the direction in which a data set is most interesting., Probability Theory and Related Fields, 80:51–77, 1988.
  • [10] M. Hristache, A. Juditski, J. Polzehl, and V. Spokoiny. Structure adaptive approach for dimension reduction., Annals of Statistics, 29:595–623, 2001.
  • [11] Peter J. Huber. Projection pursuit., The Annals of Statistics, 13(2):435–475, 1985.
  • [12] H. Ichimura. Semiparametric least squares (sls) and weighted sls estimation of single-index models., J Econometrics, 58:71–120, 1993.
  • [13] Lee K. Jones. On a conjecture of huber concerning the convergence of projection pursuit regression., Ann. Statist, 15(2):880–882, 1987.
  • [14] M.R. Kosorok., Introduction to Empirical Processes and Semiparametric Inference. Springer in Statistics, 2005.
  • [15] S. Mendelson. Learning without concentration., arXiv :1401.0304, 2014.
  • [16] Whitney K Newey. Convergence rates and asymptotic normality for series estimators., Journal of Econometrics, 79(1):147–168, 1997.
  • [17] Jammes L. Powell, James H. Stock, and Thomas M. Stoker. Semiparametric estimation of index coefficients., Econometrica, 57(6) :1403–1430, 1989.
  • [18] Xiaotong Shen. On methods of sieves and penalization., Ann. Statist., 25(6) :2555–2591, 1997.
  • [19] Vladimir Spokoiny. Parametric estimation. Finite sample theory., Ann. Statist., 40(6) :2877–2909, 2012.
  • [20] C. J. Stone. Optimal rates of convergence for nonparametric estimators., Ann. Statist., 8(6) :1348–1360, 1980.
  • [21] M. Talagrand. Majorizing measures: the generic chaining., Ann. Statist., 24(3) :1049–1103, 1996.
  • [22] J. A. Tropp. User-friendly tail bounds for sums of random matrices., Foundations of Computational Mathematics, 12:389–434, 2012.
  • [23] Yingcun Xia. Asymptotic distributions for two estimators of the single-index model., Econometric Theory, 22 :1112–1137, 2006.
  • [24] Yingcun Xia, H. Tong, W.K. Li, and L. Zhu. An adaptive estimation of dimension reduction space., Journal of the Royal Statistical Society, pages 363–410, 2002.