- Volume 25, Number 3 (2019), 1939-1976.
Hybrid regularisation and the (in)admissibility of ridge regression in infinite dimensional Hilbert spaces
We consider the problem of estimating the slope function in a functional regression with a scalar response and a functional covariate. This central problem of functional data analysis is well known to be ill-posed, thus requiring a regularised estimation procedure. The two most commonly used approaches are based on spectral truncation or Tikhonov regularisation of the empirical covariance operator. In principle, Tikhonov regularisation is the more canonical choice. Compared to spectral truncation, it is robust to eigenvalue ties, while it attains the optimal minimax rate of convergence in the mean squared sense, and not just in a concentration probability sense. In this paper, we show that, surprisingly, one can strictly improve upon the performance of the Tikhonov estimator in finite samples by means of a linear estimator, while retaining its stability and asymptotic properties by combining it with a form of spectral truncation. Specifically, we construct an estimator that additively decomposes the functional covariate by projecting it onto two orthogonal subspaces defined via functional PCA; it then applies Tikhonov regularisation to the one component, while leaving the other component unregularised. We prove that when the covariate is Gaussian, this hybrid estimator uniformly improves upon the MSE of the Tikhonov estimator in a non-asymptotic sense, effectively rendering it inadmissible. This domination is shown to also persist under discrete observation of the covariate function. The hybrid estimator is linear, straightforward to construct in practice, and with no computational overhead relative to the standard regularisation methods. By means of simulation, it is shown to furnish sizeable gains even for modest sample sizes.
Bernoulli, Volume 25, Number 3 (2019), 1939-1976.
Received: September 2017
First available in Project Euclid: 12 June 2019
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
admissibility condition index functional data analysis ill-posed problem mean integrated squared error principal component analysis rate of convergence ridge regression spectral truncation Tikhonov regularisation
Chakraborty, Anirvan; Panaretos, Victor M. Hybrid regularisation and the (in)admissibility of ridge regression in infinite dimensional Hilbert spaces. Bernoulli 25 (2019), no. 3, 1939--1976. doi:10.3150/18-BEJ1041. https://projecteuclid.org/euclid.bj/1560326433
- Supplement to “Hybrid regularisation and the (in)admissibility of ridge regression in infinite dimensional Hilbert spaces”. A companion supplement contains the proof of Theorem 1, the verification of Assumption (A3$'$) for the case of standard Brownian motion, the results of a simulation study for sample sizes $n=50$ and $n=300$ as well as the case when the functional covariate is observed with error, and comparative results when all three regularisation methods are applied to real data set.