Prediction bounds for higher order total variation regularized least squares

Abstract

We establish adaptive results for trend filtering: least squares estimation with a penalty on the total variation of $(\mathit{k}-1)$th order differences. Our approach is based on combining a general oracle inequality for the ${\ell }_1$-penalized least squares estimator with “interpolating vectors” to upper bound the “effective sparsity.” This allows one to show that the ${\ell }_1$-penalty on the kth order differences leads to an estimator that can adapt to the number of jumps in the $(\mathit{k}-1)$th order differences of the underlying signal or an approximation thereof. We show the result for $\mathit{k}\in \{1,2,3,4\}$ and indicate how it could be derived for general $\mathit{k}\in \mathbb{N}$.