Variable selection and control in least squares problems

Abstract

The classical technique of stepwise regression provides a paridigm for variable selection in the linear least squares problem. Trust region methods which control the size of the correction to the current solution estimate prove attractive for nonlinear least squares problems because of their good global convergence behaviour. Recently there has been a convergence of these techniques with the realisation that the $l_1$ trust region method also provides a form of variable selection. These results are reviewed here, and computational methods discussed.