Open Access
October 2020 Asymptotic risk and phase transition of $l_{1}$-penalized robust estimator
Hanwen Huang
Ann. Statist. 48(5): 3090-3111 (October 2020). DOI: 10.1214/19-AOS1923

Abstract

Mean square error (MSE) of the estimator can be used to evaluate the performance of a regression model. In this paper, we derive the asymptotic MSE of $l_{1}$-penalized robust estimators in the limit of both sample size $n$ and dimension $p$ going to infinity with fixed ratio $n/p\rightarrow \delta $. We focus on the $l_{1}$-penalized least absolute deviation and $l_{1}$-penalized Huber’s regressions. Our analytic study shows the appearance of a sharp phase transition in the two-dimensional sparsity-undersampling phase space. We derive the explicit formula of the phase boundary. Remarkably, the phase boundary is identical to the phase transition curve of LASSO which is also identical to the previously known Donoho–Tanner phase transition for sparse recovery. Our derivation is based on the asymptotic analysis of the generalized approximation passing (GAMP) algorithm. We establish the asymptotic MSE of the $l_{1}$-penalized robust estimator by connecting it to the asymptotic MSE of the corresponding GAMP estimator. Our results provide some theoretical insight into the high-dimensional regression methods. Extensive computational experiments have been conducted to validate the correctness of our analytic results. We obtain fairly good agreement between theoretical prediction and numerical simulations on finite-size systems.

Citation

Download Citation

Hanwen Huang. "Asymptotic risk and phase transition of $l_{1}$-penalized robust estimator." Ann. Statist. 48 (5) 3090 - 3111, October 2020. https://doi.org/10.1214/19-AOS1923

Information

Received: 1 December 2018; Revised: 1 October 2019; Published: October 2020
First available in Project Euclid: 19 September 2020

MathSciNet: MR4152636
Digital Object Identifier: 10.1214/19-AOS1923

Subjects:
Primary: 62J05 , 62J07
Secondary: 62H12

Keywords: Mean square error , minimax , penalized , phase transition , robust

Rights: Copyright © 2020 Institute of Mathematical Statistics

Vol.48 • No. 5 • October 2020
Back to Top