Statistical inference in sparse high-dimensional additive models

Abstract

In this paper, we discuss the estimation of a nonparametric component ${\mathit{f}}_1$ of a nonparametric additive model $\mathit{Y}={\mathit{f}}_1({\mathit{X}}_1)+\cdots +{\mathit{f}}_{\mathit{q}}({\mathit{X}}_{\mathit{q}})+\mathit{ϵ}$. We allow the number q of additive components to grow to infinity and we make sparsity assumptions about the number of nonzero additive components. We compare this estimation problem with that of estimating ${\mathit{f}}_1$ in the oracle model $\mathit{Z}={\mathit{f}}_1({\mathit{X}}_1)+\mathit{ϵ}$, for which the additive components ${\mathit{f}}_2,\dots ,{\mathit{f}}_{\mathit{q}}$ are known. We construct a two-step presmoothing-and-resmoothing estimator of ${\mathit{f}}_1$ and state finite-sample bounds for the difference between our estimator and a corresponding smoothing estimator ${\hat{\mathit{f}}}_1^{\text{(oracle)}}$ in the oracle model. In an asymptotic setting, these bounds can be used to show asymptotic equivalence of our estimator and the oracle estimator; the paper thus shows that, asymptotically, under strong enough sparsity conditions, knowledge of ${\mathit{f}}_2,\dots ,{\mathit{f}}_{\mathit{q}}$ has no effect on estimation accuracy. Our first step is to estimate ${\mathit{f}}_1$ with an undersmoothed estimator based on near-orthogonal projections with a group Lasso bias correction. In the second step, we construct pseudo responses $\hat{\mathit{Y}}$ by evaluating this undersmoothed estimator of ${\mathit{f}}_1$ at the design points and then apply the smoothing method of the oracle estimator ${\hat{\mathit{f}}}_1^{\text{(oracle)}}$ to the nonparametric regression problem with “responses” $\hat{\mathit{Y}}$ and covariates ${\mathit{X}}_1$. Our mathematical exposition centers primarily on establishing properties of the presmoothing estimator. We present simulation results demonstrating close-to-oracle performance of our estimator in practical applications.