The Annals of Statistics

Partially linear additive quantile regression in ultra-high dimension

Ben Sherwood and Lan Wang

Full-text: Open access

Enhanced PDF (321 KB)

Abstract

We consider a flexible semiparametric quantile regression model for analyzing high dimensional heterogeneous data. This model has several appealing features: (1) By considering different conditional quantiles, we may obtain a more complete picture of the conditional distribution of a response variable given high dimensional covariates. (2) The sparsity level is allowed to be different at different quantile levels. (3) The partially linear additive structure accommodates nonlinearity and circumvents the curse of dimensionality. (4) It is naturally robust to heavy-tailed distributions. In this paper, we approximate the nonlinear components using B-spline basis functions. We first study estimation under this model when the nonzero components are known in advance and the number of covariates in the linear part diverges. We then investigate a nonconvex penalized estimator for simultaneous variable selection and estimation. We derive its oracle property for a general class of nonconvex penalty functions in the presence of ultra-high dimensional covariates under relaxed conditions. To tackle the challenges of nonsmooth loss function, nonconvex penalty function and the presence of nonlinear components, we combine a recently developed convex-differencing method with modern empirical process techniques. Monte Carlo simulations and an application to a microarray study demonstrate the effectiveness of the proposed method. We also discuss how the method for a single quantile of interest can be extended to simultaneous variable selection and estimation at multiple quantiles.

Article information

Source
Ann. Statist., Volume 44, Number 1 (2016), 288-317.

Dates
Received: September 2014
Revised: July 2015
First available in Project Euclid: 10 December 2015

Permanent link to this document
https://projecteuclid.org/euclid.aos/1449755964

Digital Object Identifier
doi:10.1214/15-AOS1367

Mathematical Reviews number (MathSciNet)
MR3449769

Zentralblatt MATH identifier
1331.62264

Subjects
Primary: 62G35: Robustness
Secondary: 62G20: Asymptotic properties

Keywords
Quantile regression high dimensional data nonconvex penalty partial linear variable selection

Citation

Sherwood, Ben; Wang, Lan. Partially linear additive quantile regression in ultra-high dimension. Ann. Statist. 44 (2016), no. 1, 288--317. doi:10.1214/15-AOS1367. https://projecteuclid.org/euclid.aos/1449755964


Export citation

References

  • Bai, Z. D. and Wu, Y. (1994). Limiting behavior of $M$-estimators of regression coefficients in high-dimensional linear models. I. Scale-dependent case. J. Multivariate Anal. 51 211–239.
    Mathematical Reviews (MathSciNet): MR1321295
    Zentralblatt MATH: 0816.62025
    Digital Object Identifier: doi:10.1006/jmva.1994.1059
  • Belloni, A. and Chernozhukov, V. (2011). $\ell_{1}$-penalized quantile regression in high-dimensional sparse models. Ann. Statist. 39 82–130.
    Mathematical Reviews (MathSciNet): MR2797841
    Digital Object Identifier: doi:10.1214/10-AOS827
    Project Euclid: euclid.aos/1291388370
  • Bunea, F. (2004). Consistent covariate selection and post model selection inference in semiparametric regression. Ann. Statist. 32 898–927.
    Mathematical Reviews (MathSciNet): MR2065193
    Zentralblatt MATH: 1092.62045
    Digital Object Identifier: doi:10.1214/009053604000000247
    Project Euclid: euclid.aos/1085408490
  • Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360.
    Mathematical Reviews (MathSciNet): MR1946581
    Zentralblatt MATH: 1073.62547
    Digital Object Identifier: doi:10.1198/016214501753382273
  • Gilliam, M., Rifas-Shiman, S., Berkey, C., Field, A. and Colditz, G. (2003). Maternal gestational diabetes, birth weight and adolescent obesity. Pediatrics 111 221–226.
  • Greenshtein, E. and Ritov, Y. (2004). Persistence in high-dimensional linear predictor selection and the virtue of overparametrization. Bernoulli 10 971–988.
    Mathematical Reviews (MathSciNet): MR2108039
    Digital Object Identifier: doi:10.3150/bj/1106314846
    Project Euclid: euclid.bj/1106314846
  • He, X. and Shao, Q.-M. (2000). On parameters of increasing dimensions. J. Multivariate Anal. 73 120–135.
    Mathematical Reviews (MathSciNet): MR1766124
    Zentralblatt MATH: 0948.62013
    Digital Object Identifier: doi:10.1006/jmva.1999.1873
  • He, X. and Shi, P. (1996). Bivariate tensor-product $B$-splines in a partly linear model. J. Multivariate Anal. 58 162–181.
    Mathematical Reviews (MathSciNet): MR1405586
    Zentralblatt MATH: 0865.62027
    Digital Object Identifier: doi:10.1006/jmva.1996.0045
  • He, X., Wang, L. and Hong, H. G. (2013). Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data. Ann. Statist. 41 342–369.
    Mathematical Reviews (MathSciNet): MR3059421
    Digital Object Identifier: doi:10.1214/13-AOS1087
    Project Euclid: euclid.aos/1364302746
  • He, X., Zhu, Z.-Y. and Fung, W.-K. (2002). Estimation in a semiparametric model for longitudinal data with unspecified dependence structure. Biometrika 89 579–590.
    Mathematical Reviews (MathSciNet): MR1929164
    Digital Object Identifier: doi:10.1093/biomet/89.3.579
  • Huang, J., Breheny, P. and Ma, S. (2012). A selective review of group selection in high-dimensional models. Statist. Sci. 27 481–499.
    Mathematical Reviews (MathSciNet): MR3025130
    Digital Object Identifier: doi:10.1214/12-STS392
    Project Euclid: euclid.ss/1356098552
  • Huang, J., Horowitz, J. L. and Wei, F. (2010). Variable selection in nonparametric additive models. Ann. Statist. 38 2282–2313.
    Mathematical Reviews (MathSciNet): MR2676890
    Zentralblatt MATH: 1202.62051
    Digital Object Identifier: doi:10.1214/09-AOS781
    Project Euclid: euclid.aos/1278861249
  • Huang, J., Wei, F. and Ma, S. (2012). Semiparametric regression pursuit. Statist. Sinica 22 1403–1426.
    Mathematical Reviews (MathSciNet): MR3027093
    Zentralblatt MATH: 1253.62024
  • Ishida, M., Monk, D., Duncan, A. J., Abu-Amero, S., Chong, J., Ring, S. M., Pembrey, M. E., Hindmarsh, P. C., Whittaker, J. C., Stanier, P. and Moore, G. E. (2012). Maternal inheritance of a promoter variant in the imprinted PHLDA2 gene significantly increases birth weight. Am. J. Hum. Genet. 90 715–719.
  • Kai, B., Li, R. and Zou, H. (2011). New efficient estimation and variable selection methods for semiparametric varying-coefficient partially linear models. Ann. Statist. 39 305–332.
    Mathematical Reviews (MathSciNet): MR2797848
    Zentralblatt MATH: 1209.62074
    Digital Object Identifier: doi:10.1214/10-AOS842
    Project Euclid: euclid.aos/1291388377
  • Lam, C. and Fan, J. (2008). Profile-kernel likelihood inference with diverging number of parameters. Ann. Statist. 36 2232–2260.
    Mathematical Reviews (MathSciNet): MR2458186
    Zentralblatt MATH: 1209.62064
    Digital Object Identifier: doi:10.1214/07-AOS544
    Project Euclid: euclid.aos/1223908091
  • Lee, E. R., Noh, H. and Park, B. U. (2014). Model selection via Bayesian information criterion for quantile regression models. J. Amer. Statist. Assoc. 109 216–229.
    Mathematical Reviews (MathSciNet): MR3180558
    Digital Object Identifier: doi:10.1080/01621459.2013.836975
  • Li, G., Xue, L. and Lian, H. (2011). Semi-varying coefficient models with a diverging number of components. J. Multivariate Anal. 102 1166–1174.
    Mathematical Reviews (MathSciNet): MR2805656
    Zentralblatt MATH: 1216.62060
    Digital Object Identifier: doi:10.1016/j.jmva.2011.03.010
  • Lian, H., Liang, H. and Ruppert, D. (2015). Separation of covariates into nonparametric and parametric parts in high-dimensional partially linear additive models. Statist. Sinica 25 591–607.
    Mathematical Reviews (MathSciNet): MR3379090
  • Liang, H. and Li, R. (2009). Variable selection for partially linear models with measurement errors. J. Amer. Statist. Assoc. 104 234–248.
    Mathematical Reviews (MathSciNet): MR2504375
    Digital Object Identifier: doi:10.1198/jasa.2009.0127
  • Liu, X., Wang, L. and Liang, H. (2011). Estimation and variable selection for semiparametric additive partial linear models. Statist. Sinica 21 1225–1248.
    Mathematical Reviews (MathSciNet): MR2827522
    Zentralblatt MATH: 1223.62020
    Digital Object Identifier: doi:10.5705/ss.2009.140
  • Liu, Y. and Wu, Y. (2011). Simultaneous multiple non-crossing quantile regression estimation using kernel constraints. J. Nonparametr. Stat. 23 415–437.
    Mathematical Reviews (MathSciNet): MR2801302
    Zentralblatt MATH: 05930616
    Digital Object Identifier: doi:10.1080/10485252.2010.537336
  • Schumaker, L. L. (1981). Spline Functions: Basic Theory. Wiley, New York.
    Mathematical Reviews (MathSciNet): MR606200
  • Sherwood, B. and Wang, L. (2015). Supplement to “Partially linear additive quantile regression in ultra-high dimension.” DOI:10.1214/15-AOS1367SUPP.
  • Stone, C. J. (1985). Additive regression and other nonparametric models. Ann. Statist. 13 689–705.
    Mathematical Reviews (MathSciNet): MR790566
    Zentralblatt MATH: 0605.62065
    Digital Object Identifier: doi:10.1214/aos/1176349548
    Project Euclid: euclid.aos/1176349548
  • Tang, Y., Song, X., Wang, H. J. and Zhu, Z. (2013). Variable selection in high-dimensional quantile varying coefficient models. J. Multivariate Anal. 122 115–132.
    Mathematical Reviews (MathSciNet): MR3189311
    Zentralblatt MATH: 1279.62049
    Digital Object Identifier: doi:10.1016/j.jmva.2013.07.015
  • Tao, P. D. and An, L. T. H. (1997). Convex analysis approach to d.c. programming: Theory, algorithms and applications. Acta Math. Vietnam. 22 289–355.
    Mathematical Reviews (MathSciNet): MR1479751
    Zentralblatt MATH: 0895.90152
  • Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267–288.
    Mathematical Reviews (MathSciNet): MR1379242
  • Turan, N., Ghalwash, M., Kataril, S., Coutifaris, C., Obradovic, Z. and Sapienza, C. (2012). DNA methylation differences at growth related genes correlate with birth weight: A molecular signature linked to developmental origins of adult disease? BMC Medical Genomics 5 10.
  • Votavova, H., Dostalova Merkerova, M., Fejglova, K., Vasikova, A., Krejcik, Z., Pastorkova, A., Tabashidze, N., Topinka, J., Veleminsky, M., Jr., Sram, R. J. and Brdicka, R. (2011). Transcriptome alterations in maternal and fetal cells induced by tobacco smoke. Placenta 32 763–770.
  • Wang, L., Wu, Y. and Li, R. (2012). Quantile regression for analyzing heterogeneity in ultra-high dimension. J. Amer. Statist. Assoc. 107 214–222.
    Mathematical Reviews (MathSciNet): MR2949353
    Zentralblatt MATH: 1274.62289
    Digital Object Identifier: doi:10.1080/01621459.2012.656014
  • Wang, H. and Xia, Y. (2009). Shrinkage estimation of the varying coefficient model. J. Amer. Statist. Assoc. 104 747–757.
    Mathematical Reviews (MathSciNet): MR2541592
    Digital Object Identifier: doi:10.1198/jasa.2009.0138
  • Wang, H. J., Zhu, Z. and Zhou, J. (2009). Quantile regression in partially linear varying coefficient models. Ann. Statist. 37 3841–3866.
    Mathematical Reviews (MathSciNet): MR2572445
    Zentralblatt MATH: 1191.62077
    Digital Object Identifier: doi:10.1214/09-AOS695
    Project Euclid: euclid.aos/1256303529
  • Wang, L., Liu, X., Liang, H. and Carroll, R. J. (2011). Estimation and variable selection for generalized additive partial linear models. Ann. Statist. 39 1827–1851.
    Mathematical Reviews (MathSciNet): MR2893854
    Zentralblatt MATH: 1227.62053
    Digital Object Identifier: doi:10.1214/11-AOS885
    Project Euclid: euclid.aos/1311688537
  • Wei, Y. and He, X. (2006). Conditional growth charts. Ann. Statist. 34 2069–2131. With discussions and a rejoinder by the authors.
    Mathematical Reviews (MathSciNet): MR2291494
    Zentralblatt MATH: 1106.62049
    Digital Object Identifier: doi:10.1214/009053606000000623
    Project Euclid: euclid.aos/1169571786
  • Welsh, A. H. (1989). On $M$-processes and $M$-estimation. Ann. Statist. 17 337–361.
    Mathematical Reviews (MathSciNet): MR981455
    Zentralblatt MATH: 0701.62074
    Digital Object Identifier: doi:10.1214/aos/1176347021
    Project Euclid: euclid.aos/1176347021
  • Xie, H. and Huang, J. (2009). SCAD-penalized regression in high-dimensional partially linear models. Ann. Statist. 37 673–696.
    Mathematical Reviews (MathSciNet): MR2502647
    Zentralblatt MATH: 1162.62037
    Digital Object Identifier: doi:10.1214/07-AOS580
    Project Euclid: euclid.aos/1236693146
  • Xue, L. and Yang, L. (2006). Additive coefficient modeling via polynomial spline. Statist. Sinica 16 1423–1446.
    Mathematical Reviews (MathSciNet): MR2327498
    Zentralblatt MATH: 1109.62030
  • Yuan, M. and Lin, Y. (2006). Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B. Stat. Methodol. 68 49–67.
    Mathematical Reviews (MathSciNet): MR2212574
    Zentralblatt MATH: 1141.62030
    Digital Object Identifier: doi:10.1111/j.1467-9868.2005.00532.x
  • Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty. Ann. Statist. 38 894–942.
    Mathematical Reviews (MathSciNet): MR2604701
    Zentralblatt MATH: 1183.62120
    Digital Object Identifier: doi:10.1214/09-AOS729
    Project Euclid: euclid.aos/1266586618
  • Zhang, H. H., Cheng, G. and Liu, Y. (2011). Linear or nonlinear? Automatic structure discovery for partially linear models. J. Amer. Statist. Assoc. 106 1099–1112.
    Mathematical Reviews (MathSciNet): MR2894767
    Zentralblatt MATH: 1229.62051
    Digital Object Identifier: doi:10.1198/jasa.2011.tm10281
  • Zou, H. and Li, R. (2008). One-step sparse estimates in nonconcave penalized likelihood models. Ann. Statist. 36 1509–1533.
    Mathematical Reviews (MathSciNet): MR2435443
    Digital Object Identifier: doi:10.1214/009053607000000802
    Project Euclid: euclid.aos/1216237287
  • Zou, H. and Yuan, M. (2008). Regularized simultaneous model selection in multiple quantiles regression. Comput. Statist. Data Anal. 52 5296–5304.
    Mathematical Reviews (MathSciNet): MR2526595

Supplemental materials

  • Supplemental Material to “Partially linear additive quantile regression in ultra-high dimension”. We provide technical details for some of the proofs and additional simulation results.
    Digital Object Identifier: doi:10.1214/15-AOS1367SUPP
    Supplemental PDF (280 KB)

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more