December 2023 Gaussian process regression in the flat limit
Simon Barthelmé, Pierre-Olivier Amblard, Nicolas Tremblay, Konstantin Usevich
Author Affiliations +
Ann. Statist. 51(6): 2471-2505 (December 2023). DOI: 10.1214/23-AOS2336


Gaussian process (GP) regression is a fundamental tool in Bayesian statistics. It is also known as kriging and is the Bayesian counterpart to the frequentist kernel ridge regression. Most of the theoretical work on GP regression has focused on a large-n asymptotics, characterising the behaviour of GP regression as the amount of data increases. Fixed-sample analysis is much more difficult outside of simple cases, such as locations on a regular grid.

In this work, we perform a fixed-sample analysis that was first studied in the context of approximation theory by Fornberg and Driscoll (2002), called the “flat limit”. In flat-limit asymptotics, the goal is to characterise kernel methods as the length-scale of the kernel function tends to infinity, so that kernels appear flat over the range of the data. Surprisingly, this limit is well-defined, and displays interesting behaviour: Driscoll and Fornberg showed that radial basis interpolation converges in the flat limit to polynomial interpolation, if the kernel is Gaussian. Subsequent work showed that this holds true in the multivariate setting as well, but that kernels other than the Gaussian may have (polyharmonic) splines as the limit interpolant.

Leveraging recent results on the spectral behaviour of kernel matrices in the flat limit, we study the flat limit of Gaussian process regression. Results show that Gaussian process regression tends in the flat limit to (multivariate) polynomial regression, or (polyharmonic) spline regression, depending on the kernel. Importantly, this holds for both the predictive mean and the predictive variance, so that the posterior predictive distributions become equivalent.

For the proof, we introduce the notion of prediction-equivalence of semiparametric models, which lets us state flat-limit results in a compact and unified manner. Our results have practical consequences: for instance, they show that optimal GP predictions in the sense of leave-one-out loss may occur at very large length-scales, which would be invisible to current implementations because of numerical difficulties.

Funding Statement

This work was supported by MIAI@Grenoble Alpes (ANR-19-P3IA-0003), ANR project GRANOLA (ANR-21-CE48-0009) and LeafLet (ANR-19-CE23-0021), LabEx PERSYVAL-Lab (ANR-11-LABX-0025-01)


Authors 1, 2 and 3 are also affiliated with Université Grenoble Alpes and Grenoble INP. Author 4 is also affiliated with Université de Lorraine.


Download Citation

Simon Barthelmé. Pierre-Olivier Amblard. Nicolas Tremblay. Konstantin Usevich. "Gaussian process regression in the flat limit." Ann. Statist. 51 (6) 2471 - 2505, December 2023.


Received: 1 December 2021; Revised: 1 October 2023; Published: December 2023
First available in Project Euclid: 20 December 2023

MathSciNet: MR4682705
zbMATH: 07783623
Digital Object Identifier: 10.1214/23-AOS2336

Primary: 62G08
Secondary: 65D10

Keywords: flat limit , Gaussian processes , multivariate polynomials , splines

Rights: Copyright © 2023 Institute of Mathematical Statistics


This article is only available to subscribers.
It is not available for individual sale.

Vol.51 • No. 6 • December 2023
Back to Top