Open Access
April 2007 When do stepwise algorithms meet subset selection criteria?
Xiaoming Huo, Xuelei (Sherry) Ni
Ann. Statist. 35(2): 870-887 (April 2007). DOI: 10.1214/009053606000001334


Recent results in homotopy and solution paths demonstrate that certain well-designed greedy algorithms, with a range of values of the algorithmic parameter, can provide solution paths to a sequence of convex optimization problems. On the other hand, in regression many existing criteria in subset selection (including Cp, AIC, BIC, MDL, RIC, etc.) involve optimizing an objective function that contains a counting measure. The two optimization problems are formulated as (P1) and (P0) in the present paper. The latter is generally combinatoric and has been proven to be NP-hard. We study the conditions under which the two optimization problems have common solutions. Hence, in these situations a stepwise algorithm can be used to solve the seemingly unsolvable problem. Our main result is motivated by recent work in sparse representation, while two others emerge from different angles: a direct analysis of sufficiency and necessity and a condition on the mostly correlated covariates. An extreme example connected with least angle regression is of independent interest.


Download Citation

Xiaoming Huo. Xuelei (Sherry) Ni. "When do stepwise algorithms meet subset selection criteria?." Ann. Statist. 35 (2) 870 - 887, April 2007.


Published: April 2007
First available in Project Euclid: 5 July 2007

zbMATH: 1125.62079
MathSciNet: MR2336872
Digital Object Identifier: 10.1214/009053606000001334

Primary: 62J07

Keywords: concurrent optimal subset , Convex optimization , Model selection , stepwise algorithms , subset selection

Rights: Copyright © 2007 Institute of Mathematical Statistics

Vol.35 • No. 2 • April 2007
Back to Top