The Annals of Statistics

A two-stage hybrid procedure for estimating an inverse regression function

Runlong Tang, Moulinath Banerjee, and George Michailidis

Full-text: Open access


We consider a two-stage procedure (TSP) for estimating an inverse regression function at a given point, where isotonic regression is used at stage one to obtain an initial estimate and a local linear approximation in the vicinity of this estimate is used at stage two. We establish that the convergence rate of the second-stage estimate can attain the parametric n1/2 rate. Furthermore, a bootstrapped variant of TSP (BTSP) is introduced and its consistency properties studied. This variant manages to overcome the slow speed of the convergence in distribution and the estimation of the derivative of the regression function at the unknown target quantity. Finally, the finite sample performance of BTSP is studied through simulations and the method is illustrated on a data set.

Article information

Ann. Statist., Volume 39, Number 2 (2011), 956-989.

First available in Project Euclid: 8 April 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G09: Resampling methods 62G20: Asymptotic properties
Secondary: 62G07: Density estimation

Two-stage estimator bootstrap adaptive design asymptotic properties


Tang, Runlong; Banerjee, Moulinath; Michailidis, George. A two-stage hybrid procedure for estimating an inverse regression function. Ann. Statist. 39 (2011), no. 2, 956--989. doi:10.1214/10-AOS820.

Export citation


  • [1] Anbar, D. (1977). A modified Robbins–Monro procedure approximating the zero of a regression function from below. Ann. Statist. 5 229–234.
  • [2] Bambos, N. and Michailidis, G. (2004). Queueing and scheduling in random environments. Adv. in Appl. Probab. 36 293–317.
  • [3] Banerjee, M. (2007). Likelihood based inference for monotone response models. Ann. Statist. 35 931–956.
  • [4] Banerjee, M. (2009). Inference in exponential family regression models under certain shape constraints. In Advances in Multivariate Statistical Methods. Statistical Science and Interdisciplinary Research 4 249–271. World Sci. Publ., Hackensack, NJ.
  • [5] Banerjee, M. and Wellner, J. A. (2005). Confidence intervals for current status data. Scand. J. Statist. 32 405–424.
  • [6] Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and Brunk, H. D. (1972). Statistical Inference Under Order Restrictions: The Theory and Application of Isotonic Regression. Wiley, London.
  • [7] Billingsley, P. (1995). Probability and Measure, 3rd ed. Wiley, New York.
  • [8] Brunk, H. D. (1970). Estimation of isotonic regression. In Nonparametric Techniques in Statistical Inference (Proc. Sympos., Indiana Univ., Bloomington, Ind., 1969) 177–197. Cambridge Univ. Press, London.
  • [9] Durot, C. (2002). Sharp asymptotics for isotonic regression. Probab. Theory Related Fields 122 222–240.
  • [10] Durot, C. (2007). On the $\mathbbL_p$-error of monotonicity constrained estimators. Ann. Statist. 35 1080–1104.
  • [11] Durot, C. (2008). Monotone nonparametric regression with random design. Math. Methods Statist. 17 327–341.
  • [12] Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. Monographs on Statistics and Applied Probability 66. Chapman and Hall, London.
  • [13] Gasser, T., Sroka, L. and Jennen-Steinmetz, C. (1986). Residual variance and residual pattern in nonlinear regression. Biometrika 73 625–633.
  • [14] Gezmu, M. and Flournoy, N. (2006). Group up-and-down designs for dose-finding. J. Statist. Plann. Inference 136 1749–1764.
  • [15] Groeneboom, P., Hooghiemstra, G. and Lopuhaä, H. P. (1999). Asymptotic normality of the L1 error of the grenander estimator. Ann. Statist. 27 1316–1347.
  • [16] Groeneboom, P. and Wellner, J. A. (2001). Computing Chernoff’s distribution. J. Comput. Graph. Statist. 10 388–400.
  • [17] Gruet, M.-A. (1996). A nonparametric calibration analysis. Ann. Statist. 24 1474–1492.
  • [18] Hu, T. C., Móricz, F. and Taylor, R. L. (1989). Strong laws of large numbers for arrays of rowwise independent random variables. Acta Math. Hungar. 54 153–162.
  • [19] Ivanova, A., Montazer-Haghighi, A., Mohanty, S. G. and Durham, S. D. (2003). Improved up-and-down designs for phase I trials. Stat. Med. 22 69–82.
  • [20] Lai, T. L. (2003). Stochastic approximation. Ann. Statist. 31 391–406.
  • [21] Leurgans, S. (1982). Asymptotic distributions of slope-of-greatest-convex-minorant estimators. Ann. Statist. 10 287–296.
  • [22] Makowski, G. (1975). A rate of convergence for a nondecreasing regression estimator. Bull. Inst. Math. Acad. Sinica 3 61–64.
  • [23] Makowski, G. G. (1973). Laws of the iterated logarithm for permuted random variables and regression applications. Ann. Statist. 1 872–887.
  • [24] Morgan, B. J. T. (1985). The cubic logistic model for quantal assay data. Appl. Statist. 34 105–113.
  • [25] Morgan, B. J. T. (1992). Analysis of Quantal Response Data. Chapman and Hall, London.
  • [26] Müller, H.-G. and Schmitt, T. (1990). Choice of number of doses for maximum likelihood estimation of the ED50 for quantal dose-response data. Biometrics 46 117–129.
  • [27] Müller, H.-G. and Stadtmüller, U. (1987). Estimation of heteroscedasticity in regression analysis. Ann. Statist. 15 610–625.
  • [28] Osborne, C. (1991). Statistical calibration: A review. International Statistical Review 59 309–336.
  • [29] Rai, K. and Ryzin, J. V. (1981). A generalized multihit dose-response model for low-dose extrapolation. Biometrics 37 341–352.
  • [30] Robbins, H. and Monro, S. (1951). A stochastic approximation method. Ann. Math. Statist. 22 400–407.
  • [31] Robertson, T., Wright, F. T. and Dykstra, R. L. (1988). Order Restricted Statistical Inference. Wiley, Chichester.
  • [32] Rosenberger, W. F. (1996). New directions in adaptive designs. Statist. Sci. 11 137–149.
  • [33] Rosenberger, W. F. and Haines, L. M. (2002). Competing designs for phase i clinical trials: A review. Stat. Med. 21 2757–2770.
  • [34] Shiryaev, A. N. (1995). Probability, 2nd ed. Springer, New York.
  • [35] Staniswalis, J. G. and Cooper, V. (1988). Kernel estimates of dose response. Biometrics 44 1103–1119.
  • [36] Stylianou, M. and Flournoy, N. (2002). Dose finding using the biased coin up-and-down design and isotonic regression. Biometrics 58 171–177.
  • [37] Wright, F. T. (1981). The asymptotic behavior of monotone regression estimates. Ann. Statist. 9 443–448.