August 2022 A Unifying Framework of High-Dimensional Sparse Estimation with Difference-of-Convex (DC) Regularizations
Shanshan Cao, Xiaoming Huo, Jong-Shi Pang
Author Affiliations +
Statist. Sci. 37(3): 411-424 (August 2022). DOI: 10.1214/21-STS832

Abstract

Under the linear regression framework, we study the variable selection problem when the underlying model is assumed to have a small number of nonzero coefficients. Nonconvex penalties in specific forms are well studied in the literature for sparse estimation. Recent work pointed out that nearly all existing nonconvex penalties can be represented as difference-of-convex (DC) functions, which are the difference of two convex functions, while itself may not be convex. There is a large existing literature on optimization problems when their objectives and/or constraints involve DC functions. Efficient numerical solutions have been proposed. Under the DC framework, directional-stationary (d-stationary) solutions are considered, and they are usually not unique. In this paper, we show that under some mild conditions, a certain subset of d-stationary solutions in an optimization problem (with a DC objective) has some ideal statistical properties: namely, asymptotic estimation consistency, asymptotic model selection consistency, asymptotic efficiency. Our assumptions are either weaker than or comparable with those conditions that have been adopted in other existing works. This work shows that DC is a nice framework to offer a unified approach to these existing works where nonconvex penalties are involved. Our work bridges the communities of optimization and statistics.

Acknowledgments

This project is partially supported by the Transdisciplinary Research Institute for Advancing Data Science (TRIAD), http://triad.gatech.edu, which is a part of the TRIPODS program at NSF and locates at Georgia Tech, enabled by NSF Grant CCF-1740776. The first two authors are also partially supported by NSF Grants DMS-1613152 and DMS-2015363.

Citation

Download Citation

Shanshan Cao. Xiaoming Huo. Jong-Shi Pang. "A Unifying Framework of High-Dimensional Sparse Estimation with Difference-of-Convex (DC) Regularizations." Statist. Sci. 37 (3) 411 - 424, August 2022. https://doi.org/10.1214/21-STS832

Information

Published: August 2022
First available in Project Euclid: 21 June 2022

MathSciNet: MR4444375
zbMATH: 07569969
Digital Object Identifier: 10.1214/21-STS832

Keywords: (Generalized) linear regression , asymptotic optimality , DC algorithms , difference of convex (DC) functions , high-dimensional sparse estimation , model selection consistency , nonconvex regularization

Rights: Copyright © 2022 Institute of Mathematical Statistics

JOURNAL ARTICLE
14 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

Vol.37 • No. 3 • August 2022
Back to Top