The Annals of Statistics

Risk bounds in isotonic regression

Cun-Hui Zhang

Full-text: Open access


Nonasymptotic risk bounds are provided for maximum likelihood-type isotonic estimators of an unknown nondecreasing regression function, with general average loss at design points. These bounds are optimal up to scale constants, and they imply uniform $n^{-1/3}$-consistency of the $\ell_p$ risk for unknown regression functions of uniformly bounded variation, under mild assumptions on the joint probability distribution of the data, with possibly dependent observations.

Article information

Ann. Statist., Volume 30, Number 2 (2002), 528-555.

First available in Project Euclid: 14 May 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression 62G05: Estimation
Secondary: 62J02: General nonlinear regression 62G20: Asymptotic properties

Nonparametric regression isotonic regression risk bounds least squares estimator maximum likelihood estimator


Zhang, Cun-Hui. Risk bounds in isotonic regression. Ann. Statist. 30 (2002), no. 2, 528--555. doi:10.1214/aos/1021379864.

Export citation


  • AYER, M., BRUNK, H. D., EWING, G. M., REID, W. T. and SILVERMAN, E. (1955). An empirical distribution function for sampling with incomplete information. Ann. Math. Statist. 26 641-647.
  • BARLOW, R. E., BARTHOLOMEW, D. J., BREMNER, J. M. and BRUNK, H. D. (1972). Statistical Inference Under Order Restrictions. Wiley, New York.
  • BARLOW, R. E. and UBHAYA, V. A. (1971). Isotonic approximation. In Optimizing Methods in Statistics (J. S. Rustagi, ed.) 77-86. Academic Press, New York.
  • BIRGÉ, L. (1987). Estimating a density under order restrictions: Nonasymptotic minimax risk. Ann. Statist. 15 995-1012.
  • BIRGÉ, L. (1989). The Grenander estimator: A nonasymptotic approach. Ann. Statist. 17 1532-1549.
  • BIRGÉ, L. and MASSART, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113-150.
  • BRUNK, H. D. (1970). Estimation of isotonic regression (with discussion by R. Pyke). In Nonparametric Techniques in Statistical Inference (M. L. Puri, ed.) 177-197. Cambridge Univ. Press, London.
  • BRUNK, H. D. and JOHANSEN, S. (1970). A generalized Radon-Nikodym derivative. Pacific J. Math. 34 585-617.
  • CHOW, Y. S. and LAI, T. L. (1975). Some one-sided theorems on the tail distribution of sample sums with applications to the last time and largest excess of boundary crossings. Trans. Amer. Math. Soc. 208 51-72.
  • CHOW, Y. S. and LAI, T. L. (1978). Paley-type inequalities and convergence rates related to the law of large numbers and extended renewal theory. Z. Wahrsch. Verw. Gebiete 45 1-19.
  • DONOHO, D. (1991). Gel'fand n-widths and the method of least squares. Technical report, Dept. Statist., Univ. California, Berkeley.
  • GRENANDER, U. (1956). On the theory of mortality measurement II. Skand. Aktuarietidskr. 39 125- 153.
  • GRENANDER, U. (1980). Abstract Inference. Wiley, New York.
  • GROENEBOOM, P. (1983). The concave majorant of Brownian motion. Ann. Probab. 11 1016-1027.
  • GROENEBOOM, P. (1985). Estimating a monotone density. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer (L. Le Cam and R. A. Olshen, eds.) 2 539-555. Wadsworth, New York.
  • GROENEBOOM, P. and WELLNER, J. A. (1992). Information Bounds and Nonparametric Maximum Likelihood Estimation. Birkhäuser, New York.
  • HOFFMANN-JORGENSEN, J. (1974). Sums of independent Banach space valued random variables. Studia Math. 52 159-189.
  • MEYER, M. and WOODROOFE, M. (2000). On the degrees of freedom in shape-restricted regression. Ann. Statist. 28 1083-1104.
  • PRAKASA RAO, B. L. S. (1969). Estimation of a unimodal density. Sankhy¯a Ser. A 31 23-36.
  • ROBERTSON, T. and WALTMAN, P. (1968). On estimating monotone parameters. Ann. Math. Statist. 39 1030-1039.
  • ROBERTSON, T., WRIGHT, E. T. and DYKSTRA, R. L. (1988). Order Restricted Statistical Inference. Wiley, New York.
  • SERFLING, R. J. (1970). Moment inequalities for the maximum cumulative sum. Ann. Math. Statist. 41 1227-1234.
  • STEIN, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135- 1151.
  • VAN DE GEER, S. (1990). Estimating a regression function. Ann. Statist. 18 907-924.
  • VAN DE GEER, S. (1993). Hellinger consistency of certain nonparametric maximum likelihood estimators. Ann. Statist. 21 14-44.
  • VAN EEDEN, C. (1956). Maximum likelihood estimation of ordered probabilities. Indag. Math. 18 444-455.
  • VAN EEDEN, C. (1957a). Maximum likelihood estimation of partially or completely ordered parameters I. Indag. Math. 19 128-136.
  • VAN EEDEN, C. (1957b). Maximum likelihood estimation of partially or completely ordered parameters II. Indag. Math. 19 201-211.
  • WANG, Y. (1996). The L2 risk of an isotonic estimate. Comm. Statist. Theory Methods 25 281-294.