The Annals of Statistics

On the degrees of freedom in shape-restricted regression

Mary Meyer and Michael Woodroofe

Full-text: Open access


For the problem of estimating a regression function, $\mu$ say, subject to shape constraints, like monotonicity or convexity, it is argued that the divergence of the maximum likelihood estimator provides a useful measure of the effective dimension of the model. Inequalities are derived for the expected mean squared error of the maximum likelihood estimator and the expected residual sum of squares. These generalize equalities from the case of linear regression. As an application, it is shown that the maximum likelihood estimator of the error variance $\sigma^2$ is asymptotically normal with mean $\sigma^2$ and variance $2\sigma_2/n$. For monotone regression, it is shown that the maximum likelihood estimator of $\mu$ attains the optimal rate of convergence, and a bias correction to the maximum likelihood estimator of $\sigma^2$ is derived.

Article information

Ann. Statist., Volume 28, Number 4 (2000), 1083-1104.

First available in Project Euclid: 12 March 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression

Asymptotic distribution bias reduction divergence effective dimension simulation Stein's identity variance estimation


Meyer, Mary; Woodroofe, Michael. On the degrees of freedom in shape-restricted regression. Ann. Statist. 28 (2000), no. 4, 1083--1104. doi:10.1214/aos/1015956708.

Export citation


  • Breiman, L. (1968). Probability. Addison-Weseley, Reading, MA.
  • Donoho, D. (1990). Gelfand n-widths and the method of least squares. Unpublished manuscript.
  • Donoho, D. and Johnstone, I. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200-1225.
  • Efromovich, S. (1997). On quasi-linear wavelet estimation. J. Amer. Statist. Assoc. 94 189-204.
  • Feller, W. (1971). An Introduction to Probability Theory and Its Applications 2. Wiley, New York.
  • Gasser, T. Sroka, L. and Jennen-Steinmetz, C. (1986). Residual variance and residual pattern in non-linear regression. Biometrika 73 625-633.
  • Groeneboom, P. (1985). Estimating a monotone density. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer (L. Le Cam and R. Olshen, eds.) 2 539-558. Wadsworth and Brooks/Cole, Belmont, CA.
  • Groeneboom, P. (1989). Brownian motion with a parabolic drift and airy functions. Probab. Theory Related Fields 81 79-109.
  • Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman and Hall, London.
  • Klass, M. (1983). On the maximum of a random walkwith a small negative drift. Ann. Probab. 11 491-505.
  • Meyer, M. (1996). Shape restricted inference with applications to nonparametric regression, smooth nonparametric regression, and density estimation. Ph.D. thesis, Dept. statistics, Univ. Michigan.
  • Meyer, M. and Woodroofe, M. (1998). Variance estimation in monotone regression. Technical Report 322, Dept. Statistics, Univ. Michigan.
  • Rice, J. (1984). Bandwidth choice for nonparametric regression. Ann. Statist. 12 1215-1230.
  • Robertson, T., Wright, F. and Dykstra, R. (1988). Order Restricted Inference, Wiley, New York.
  • Stein, C. (1981). Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 1135-1151.
  • Van der Geer, S. (1990). Estimating a regression function. Ann. Statist. 18 907-924.
  • Woodroofe, M. (1982). Non-linear Renewal Theory in Sequential Analysis. SIAM, Philadelphia.
  • Wright, F. T. (1981). The asymptotic behavior of monotone regression estimates. Ann. Statist. 9 449-453.