The Annals of Statistics

Variable selection with Hamming loss

Cristina Butucea, Mohamed Ndaoud, Natalia A. Stepanova, and Alexandre B. Tsybakov

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

We derive nonasymptotic bounds for the minimax risk of variable selection under expected Hamming loss in the Gaussian mean model in $\mathbb{R}^{d}$ for classes of at most $s$-sparse vectors separated from 0 by a constant $a>0$. In some cases, we get exact expressions for the nonasymptotic minimax risk as a function of $d,s,a$ and find explicitly the minimax selectors. These results are extended to dependent or non-Gaussian observations and to the problem of crowdsourcing. Analogous conclusions are obtained for the probability of wrong recovery of the sparsity pattern. As corollaries, we derive necessary and sufficient conditions for such asymptotic properties as almost full recovery and exact recovery. Moreover, we propose data-driven selectors that provide almost full and exact recovery adaptively to the parameters of the classes.

Article information

Source
Ann. Statist., Volume 46, Number 5 (2018), 1837-1875.

Dates
Received: December 2015
Revised: March 2017
First available in Project Euclid: 17 August 2018

Permanent link to this document
https://projecteuclid.org/euclid.aos/1534492821

Digital Object Identifier
doi:10.1214/17-AOS1572

Mathematical Reviews number (MathSciNet)
MR3845003

Zentralblatt MATH identifier
06964318

Subjects
Primary: 62G05: Estimation 62G08: Nonparametric regression 62G20: Asymptotic properties

Keywords
Adaptive variable selection almost full recovery exact recovery Hamming loss minimax selectors nonasymptotic minimax selection bounds phase transitions

Citation

Butucea, Cristina; Ndaoud, Mohamed; Stepanova, Natalia A.; Tsybakov, Alexandre B. Variable selection with Hamming loss. Ann. Statist. 46 (2018), no. 5, 1837--1875. doi:10.1214/17-AOS1572. https://projecteuclid.org/euclid.aos/1534492821


Export citation

References

  • [1] Abramovich, F. and Benjamini, Y. (1995). Thresholding of wavelet coefficients as multiple hypotheses testing procedure. In Wavelets and Statistics, Lecture Notes in Statistics 103 5–14. Springer, New York.
    Mathematical Reviews (MathSciNet): MR1364677
    Zentralblatt MATH: 0875.62081
  • [2] Abramovich, F., Benjamini, Y., Donoho, D. L. and Johnstone, I. M. (2006). Adapting to unknown sparsity by controlling the false discovery rate. Ann. Statist. 34 584–653.
    Mathematical Reviews (MathSciNet): MR2281879
    Zentralblatt MATH: 1092.62005
    Digital Object Identifier: doi:10.1214/009053606000000074
    Project Euclid: euclid.aos/1151418235
  • [3] Abramowitz, M. and Stegun, I. A. (1964). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. National Bureau of Standards Applied Mathematics Series 55. For sale by the Superintendent of Documents, U.S. Government Printing Office, Washington, D.C.
    Zentralblatt MATH: 0171.38503
  • [4] Arias-Castro, E. and Chen, S. (2017). Distribution-free multiple testing. Electron. J. Stat. 11 1983–2001.
    Zentralblatt MATH: 1361.62023
    Digital Object Identifier: doi:10.1214/17-EJS1277
  • [5] Bertin, K. and Lecué, G. (2008). Selection of variables and dimension reduction in high-dimensional non-parametric regression. Electron. J. Stat. 2 1224–1241.
    Mathematical Reviews (MathSciNet): MR2461900
    Zentralblatt MATH: 1320.62085
    Digital Object Identifier: doi:10.1214/08-EJS327
    Project Euclid: euclid.ejs/1229450668
  • [6] Bogdan, M., van den Berg, E., Sabatti, C., Su, W. and Candès, E. J. (2015). SLOPE—Adaptive variable selection via convex optimization. Ann. Appl. Stat. 9 1103–1140.
    Zentralblatt MATH: 06525980
    Digital Object Identifier: doi:10.1214/15-AOAS842
    Project Euclid: euclid.aoas/1446488733
  • [7] Butucea, C., Ingster, Y. I. and Suslina, I. A. (2015). Sharp variable selection of a sparse submatrix in a high-dimensional noisy matrix. ESAIM Probab. Stat. 19 115–134.
    Zentralblatt MATH: 1330.62169
    Digital Object Identifier: doi:10.1051/ps/2014017
  • [8] Butucea, C., Ndaoud, M., Stepanova, N. A. and Tsybakov, A. B. (2018). Supplement to “Variable selection with Hamming loss.” DOI:10.1214/17-AOS1572SUPP.
  • [9] Butucea, C. and Stepanova, N. (2017). Adaptive variable selection in nonparametric sparse additive models. Electron. J. Stat. 11 2321–2357.
    Zentralblatt MATH: 1365.62133
    Digital Object Identifier: doi:10.1214/17-EJS1275
  • [10] Collier, O., Comminges, L., Tsybakov, A. B. and Verzelen, N. (2016). Optimal adaptive estimation of linear functionals under sparsity. http://arxiv.org/abs/1611.09744.
  • [11] Comminges, L. and Dalalyan, A. S. (2012). Tight conditions for consistency of variable selection in the context of high dimensionality. Ann. Statist. 40 2667–2696.
    Zentralblatt MATH: 1373.62154
    Digital Object Identifier: doi:10.1214/12-AOS1046
    Project Euclid: euclid.aos/1359987534
  • [12] Gao, C., Lu, Y. and Zhou, D. (2016). Exact exponent in optimal rates for crowdsourcing. http://arxiv.org/abs/1605.07696.
  • [13] Genovese, C. R., Jin, J., Wasserman, L. and Yao, Z. (2012). A comparison of the lasso and marginal regression. J. Mach. Learn. Res. 13 2107–2143.
    Zentralblatt MATH: 06276177
  • [14] Hall, P. and Jin, J. (2010). Innovated higher criticism for detecting sparse signals in correlated noise. Ann. Statist. 38 1686–1732.
    Zentralblatt MATH: 1189.62080
    Digital Object Identifier: doi:10.1214/09-AOS764
    Project Euclid: euclid.aos/1269452652
  • [15] Ingster, Y. I. and Stepanova, N. A. (2014). Adaptive variable selection in nonparametric sparse regression. J. Math. Sci. 199 184–201.
    Zentralblatt MATH: 1311.62056
    Digital Object Identifier: doi:10.1007/s10958-014-1846-7
  • [16] Ji, P. and Jin, J. (2012). UPS delivers optimal phase diagram in high-dimensional variable selection. Ann. Statist. 40 73–103.
    Zentralblatt MATH: 1246.62160
    Digital Object Identifier: doi:10.1214/11-AOS947
    Project Euclid: euclid.aos/1331830775
  • [17] Jin, J., Zhang, C.-H. and Zhang, Q. (2014). Optimality of graphlet screening in high dimensional variable selection. J. Mach. Learn. Res. 15 2723–2772.
    Zentralblatt MATH: 1319.62139
  • [18] Lafferty, J. and Wasserman, L. (2008). Rodeo: Sparse, greedy nonparametric regression. Ann. Statist. 36 28–63.
    Mathematical Reviews (MathSciNet): MR2387963
    Zentralblatt MATH: 1132.62026
    Digital Object Identifier: doi:10.1214/009053607000000811
    Project Euclid: euclid.aos/1201877293
  • [19] Lehmann, E. L. and Romano, J. P. (2005). Testing Statistical Hypotheses, 3rd ed. Springer, New York.
    Zentralblatt MATH: 1076.62018
  • [20] Lounici, K. (2008). Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators. Electron. J. Stat. 2 90–102.
    Zentralblatt MATH: 1306.62155
    Digital Object Identifier: doi:10.1214/08-EJS177
  • [21] Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. Ann. Statist. 34 1436–1462.
    Mathematical Reviews (MathSciNet): MR2278363
    Zentralblatt MATH: 1113.62082
    Digital Object Identifier: doi:10.1214/009053606000000281
    Project Euclid: euclid.aos/1152540754
  • [22] Meinshausen, N. and Bühlmann, P. (2010). Stability selection. J. R. Stat. Soc. Ser. B. Stat. Methodol. 72 417–473.
  • [23] Neuvial, P. and Roquain, E. (2012). On false discovery rate thresholding for classification under sparsity. Ann. Statist. 40 2572–2600.
    Zentralblatt MATH: 1373.62315
    Digital Object Identifier: doi:10.1214/12-AOS1042
    Project Euclid: euclid.aos/1359987531
  • [24] Wainwright, M. J. (2009). Sharp thresholds for high-dimensional and noisy sparsity recovery using $\ell_{1}$-constrained quadratic programming (Lasso). IEEE Trans. Inform. Theory 55 2183–2202.
    Mathematical Reviews (MathSciNet): MR2729873
    Zentralblatt MATH: 1367.62220
    Digital Object Identifier: doi:10.1109/TIT.2009.2016018
  • [25] Wasserman, L. and Roeder, K. (2009). High-dimensional variable selection. Ann. Statist. 37 2178–2201.
    Zentralblatt MATH: 1173.62054
    Digital Object Identifier: doi:10.1214/08-AOS646
    Project Euclid: euclid.aos/1247663752
  • [26] Zhang, A. Y. and Zhou, H. H. (2016). Minimax rates of community detection in stochastic block models. Ann. Statist. 44 2252–2280.
    Zentralblatt MATH: 1355.60125
    Digital Object Identifier: doi:10.1214/15-AOS1428
    Project Euclid: euclid.aos/1473685275
  • [27] Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty. Ann. Statist. 38 894–942.
    Zentralblatt MATH: 1183.62120
    Digital Object Identifier: doi:10.1214/09-AOS729
    Project Euclid: euclid.aos/1266586618
  • [28] Zhao, P. and Yu, B. (2006). On model selection consistency of Lasso. J. Mach. Learn. Res. 7 2541–2563.
    Zentralblatt MATH: 1222.62008

Supplemental materials

  • Supplement to “Variable selection with Hamming loss”. We derive a general lower bound for the minimax risk over all selectors on the class of at most $s$-sparse vectors. The main term of this bound is a Bayes risk with arbitrary prior and the non-asymptotic remainder term is given explicitly. Using this, we prove the lower bounds of Theorems 2.2, 3.2 and 3.3.
    Digital Object Identifier: doi:10.1214/17-AOS1572SUPP
    Supplemental files are immediately available to subscribers. Non-subscribers gain access to supplemental files with the purchase of the article.

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more