Brazilian Journal of Probability and Statistics

Additive models for quantile regression: Model selection and confidence bandaids

Roger Koenker

Full-text: Open access

Abstract

Additive models for conditional quantile functions provide an attractive framework for nonparametric regression applications focused on features of the response beyond its central tendency. Total variation roughness penalities can be used to control the smoothness of the additive components much as squared Sobelev penalties are used for classical L2 smoothing splines. We describe a general approach to estimation and inference for additive models of this type. We focus attention primarily on selection of smoothing parameters and on the construction of confidence bands for the nonparametric components. Both pointwise and uniform confidence bands are introduced; the uniform bands are based on the Hotelling [Amer. J. Math. 61 (1939) 440–460] tube approach. Some simulation evidence is presented to evaluate finite sample performance and the methods are also illustrated with an application to modeling childhood malnutrition in India.

Article information

Source
Braz. J. Probab. Stat., Volume 25, Number 3 (2011), 239-262.

Dates
First available in Project Euclid: 22 August 2011

Permanent link to this document
https://projecteuclid.org/euclid.bjps/1313973394

Digital Object Identifier
doi:10.1214/10-BJPS131

Mathematical Reviews number (MathSciNet)
MR2832886

Zentralblatt MATH identifier
1236.62031

Keywords
Quantile regression additive model confidence bands Hotelling tubes

Citation

Koenker, Roger. Additive models for quantile regression: Model selection and confidence bandaids. Braz. J. Probab. Stat. 25 (2011), no. 3, 239--262. doi:10.1214/10-BJPS131. https://projecteuclid.org/euclid.bjps/1313973394


Export citation

References

  • Belloni, A. and Chernozhukov, V. (2009). L1-penalized quantile regression in high-dimensional sparse models. Available at http://arXiv.org/abs/1001.0188.
  • Breiman, L. and Friedman, J. (1985). Estimating optimal transformations for multiple regression and correlation. Journal of the American Statistical Association 80, 580–598.
  • Davies, P. L. and Kovac, A. (2001). Local extremes, runs, strings and multiresolution. Ann. Statist. 29, 1–48.
  • Delbaere, I., Vansteelandt, S., Bacquer, D. D., Verstraelen, H., Gerris, J., Sutter, P. D. and Temmerman, M. (2007). Should we adjust for gestational age when analysing birth weights? The use of z-scores revisited. Human Reproduction 22, 2080–2083.
  • Fenske, N., Kneib, T. and Hothorn, T. (2011). Identifying risk factors for severe childhood malnutrition by boosting additive quantile regression. Journal of the American Statistical Association 106, 494–510.
  • Hastie, T. and Tibshirani, R. (1986). Generalized additive models. Statistical Science 1, 297–310.
  • Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. London: Chapman and Hall.
  • Hotelling, H. (1939). Tubes and spheres in n-space and a class of statistical problems. American Journal of Mathematics 61, 440–460.
  • Johansen, S. and Johnstone, I. M. (1990). Hotelling’s theorem on the volume of tubes: Some illustrations in simultaneous inference and data analysis. The Annals of Statistics 18, 652–684.
  • Knowles, M. (1987). Simultaneous confidence bands for random functions. Ph.D. thesis, Stanford Univ.
  • Koenker, R. (2005). Quantile Regression. London: Cambridge Univ. Press.
  • Koenker, R. (2010). quantreg: Quantile Regression, v4.45. Available at http://cran.r-project.org/package=quantreg.
  • Koenker, R. and Mizera, I. (2004). Penalized triograms: Total variation regularization for bivariate smoothing. J. R. Stat. Soc. Ser. B Stat. Methodol. 66, 145–163.
  • Koenker, R. and Ng, P. (2005). A Frisch–Newton algorithm for sparse quantile regression. Mathematicae Applicatae Sinica 21, 225–236.
  • Koenker, R., Ng, P. and Portnoy, S. (1994). Quantile smoothing splines. Biometrika 81, 673–680.
  • Krivobokova, T., Kneib, T. and Claeskens, G. (2011). Simultaneous confidence bands for penalized spline estimators. Journal of the American Statistical Association 105, 852–863.
  • Loader, C. (2010). locfit: Local regression, likelihood and density estimation, v1.5-5. Available at http://cran.r-project.org/package=locfit.
  • Machado, J. (1993). Robust model selection and M-estimation. Econometric Theory 9, 478–493.
  • Meyer, M. and Woodroofe, M. (2000). On the degrees of freedom in shape-restricted regression. The Annals of Statistics 28, 1083–1104.
  • Naiman, D. (1986). Conservative confidence bands in curvilinear reression. The Annals of Statistics 14, 896–906.
  • Nychka, D. (1983). Bayesian Confidence Intervals for smoothing splines. Journal of the American Statistical Association 83, 1134–1143.
  • Pötscher, B. and Leeb, H. (2009). On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD and thresholding. Journal of Multivariate Analysis 100, 2065–2082.
  • Powell, J. L. (1991). Estimation of monotonic regression models under quantile restrictions. In Nonparametric and Semiparametric Methods in Econometrics (W. Barnett, J. Powell and G. Tauchen, eds.) 357–384. Cambridge: Cambridge Univ. Press.
  • Rudin, L., Osher, S. and Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268.
  • Ruppert, D., Wand, M. and Carroll, R. J. (2003). Semiparametric Regression. Cambridge: Cambridge Univ. Press.
  • Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics 6, 461–464.
  • Sun, J. (1993). Tail probabilities of the maxima of Gaussian random fields. The Annals of Probability 21, 34–71.
  • Sun, J. and Loader, C. (1994). Simultaneous confidence bands for linear regression and smoothing. The Annals of Statistics 22, 1328–1347.
  • Sun, J., Loader, C. and McCormick, W. (2000). Confidence bands in generalized linear models. The Annals of Statistics 28, 429–460.
  • Tibshirani, R., Saunders, M., Rosset, S., Zhu, J. and Knight K. (2005). Sparsity and smoothness via the fused lasso. Journal of the Royal Statistical Society B 67, 91–108.
  • Wahba, G. (1983). Bayesian “Confidence Intervals” for the cross-validated smoothing spline. Journal of the Royal Statistical Society B 45, 133–150.
  • Wand, M. P. and Ormerod, J. T. (2008). On semiparametric regression with O’Sullivan penalized splines. Aust. N. Z. J. Stat. 50, 179–198.
  • Weyl, H. (1939). On the Volume of Tubes. Amer. J. Math. 61, 461–472.
  • Wood, S. (2006). Generalized Additive Models: An Introduction with R. Boca Raton, FL: Chapman-Hall.
  • Wood, S. (2010). mgcv: GAMs with GCV/AIC/REML smoothness estimation and GAMMs by PQL. Available at http://cran.r-project.org/package=mgcv.