The Annals of Statistics
- Ann. Statist.
- Volume 39, Number 5 (2011), 2626-2657.
Bayesian inverse problems with Gaussian priors
B. T. Knapik, A. W. van der Vaart, and J. H. van Zanten
Full-text: Open access
Abstract
The posterior distribution in a nonparametric inverse problem is shown to contract to the true parameter at a rate that depends on the smoothness of the parameter, and the smoothness and scale of the prior. Correct combinations of these characteristics lead to the minimax rate. The frequentist coverage of credible sets is shown to depend on the combination of prior and true parameter, with smoother priors leading to zero coverage and rougher priors to conservative coverage. In the latter case credible sets are of the correct order of magnitude. The results are numerically illustrated by the problem of recovering a function from observation of a noisy version of its primitive.
Article information
Source
Ann. Statist., Volume 39, Number 5 (2011), 2626-2657.
Dates
First available in Project Euclid: 22 December 2011
Permanent link to this document
https://projecteuclid.org/euclid.aos/1324563350
Digital Object Identifier
doi:10.1214/11-AOS920
Mathematical Reviews number (MathSciNet)
MR2906881
Zentralblatt MATH identifier
1232.62079
Subjects
Primary: 62G05: Estimation 62G15: Tolerance and confidence regions
Secondary: 62G20: Asymptotic properties
Keywords
Credible set posterior distribution Gaussian prior rate of contraction
Citation
Knapik, B. T.; van der Vaart, A. W.; van Zanten, J. H. Bayesian inverse problems with Gaussian priors. Ann. Statist. 39 (2011), no. 5, 2626--2657. doi:10.1214/11-AOS920. https://projecteuclid.org/euclid.aos/1324563350
References
- [1] Bertero, M. (1986). Regularization methods for linear inverse problems. In Inverse Problems (Montecatini Terme, 1986). Lecture Notes in Math. 1225 52–112. Springer, Berlin.
- [2] Bickel, P. J., Klaassen, C. A. J., Ritov, Y. and Wellner, J. A. (1993). Efficient and Adaptive Estimation for Semiparametric Models. Johns Hopkins Univ. Press, Baltimore, MD.
- [3] Bissantz, N., Hohage, T., Munk, A. and Ruymgaart, F. (2007). Convergence rates of general regularization methods for statistical inverse problems and applications. SIAM J. Numer. Anal. 45 2610–2636 (electronic).Mathematical Reviews (MathSciNet): MR2361904
Zentralblatt MATH: 05485711
Digital Object Identifier: doi:10.1137/060651884 - [4] Bontemps, D. (2011). Bernstein–von Mises theorems for Gaussian regression with increasing number of regressors. Ann. Statist. 39 2557–2584.Zentralblatt MATH: 1231.62061
Digital Object Identifier: doi:10.1214/11-AOS912
Project Euclid: euclid.aos/1322663468 - [5] Castillo, I. (2011). A semiparametric Bernstein–von Mises theorem for Gaussian process priors. Probab. Theory Related Fields. To appear. DOI:10.1007/s00440-010-0316-5.
- [6] Cavalier, L. (2008). Nonparametric statistical inverse problems. Inverse Problems 24 034004, 19.Mathematical Reviews (MathSciNet): MR2421941
Digital Object Identifier: doi:10.1088/0266-5611/24/3/034004
Zentralblatt MATH: 1137.62323 - [7] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903–923.Mathematical Reviews (MathSciNet): MR1232525
Zentralblatt MATH: 0778.62003
Digital Object Identifier: doi:10.1214/aos/1176349157
Project Euclid: euclid.aos/1176349157 - [8] Donoho, D. L. (1994). Statistical estimation and optimal recovery. Ann. Statist. 22 238–270.Mathematical Reviews (MathSciNet): MR1272082
Zentralblatt MATH: 0805.62014
Digital Object Identifier: doi:10.1214/aos/1176325367
Project Euclid: euclid.aos/1176325367 - [9] Donoho, D. L. and Low, M. G. (1992). Renormalization exponents and optimal pointwise rates of convergence. Ann. Statist. 20 944–970.Mathematical Reviews (MathSciNet): MR1165601
Zentralblatt MATH: 0797.62032
Digital Object Identifier: doi:10.1214/aos/1176348665
Project Euclid: euclid.aos/1176348665 - [10] Feller, W. (1971). An Introduction to Probability Theory and Its Applications. Vol. II, 2nd ed. Wiley, New York.
- [11] Florens, J. and Simoni, A. (2011). Regularizing priors for linear inverse problems. Preprint.
- [12] Freedman, D. (1999). On the Bernstein–von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119–1140.Mathematical Reviews (MathSciNet): MR1740119
Zentralblatt MATH: 0957.62002
Project Euclid: euclid.aos/1017938917 - [13] Ghosal, S. (1999). Asymptotic normality of posterior distributions in high-dimensional linear models. Bernoulli 5 315–331.Mathematical Reviews (MathSciNet): MR1681701
Digital Object Identifier: doi:10.2307/3318438
Project Euclid: euclid.bj/1173147909
Zentralblatt MATH: 0948.62007 - [14] Ghosal, S. (2000). Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity. J. Multivariate Anal. 74 49–68.Mathematical Reviews (MathSciNet): MR1790613
Zentralblatt MATH: 1118.62309
Digital Object Identifier: doi:10.1006/jmva.1999.1874 - [15] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531.Mathematical Reviews (MathSciNet): MR1790007
Digital Object Identifier: doi:10.1214/aos/1016218228
Project Euclid: euclid.aos/1016218228
Zentralblatt MATH: 1105.62315 - [16] Goldenshluger, A. and Pereverzev, S. V. (2003). On adaptive inverse estimation of linear functionals in Hilbert scales. Bernoulli 9 783–807.Mathematical Reviews (MathSciNet): MR2047686
Digital Object Identifier: doi:10.3150/bj/1066418878
Project Euclid: euclid.bj/1066418878
Zentralblatt MATH: 1055.62034 - [17] Halmos, P. R. (1967). A Hilbert Space Problem Book. Van Nostrand, Princeton.
- [18] Leahu, H. (2011). On the Bernstein–von Mises phenomenon in the Gaussian white noise model. Electron. J. Stat. 5 373–404.
- [19] Mair, B. A. and Ruymgaart, F. H. (1996). Statistical inverse estimation in Hilbert scales. SIAM J. Appl. Math. 56 1424–1444.Mathematical Reviews (MathSciNet): MR1409127
Zentralblatt MATH: 0864.62020
Digital Object Identifier: doi:10.1137/S0036139994264476 - [20] Mandelbaum, A. (1984). Linear estimators and measurable linear transformations on a Hilbert space. Z. Wahrsch. Verw. Gebiete 65 385–397.
- [21] Mathé, P. and Pereverzev, S. V. (2006). Regularization of some linear ill-posed problems with discretized random noisy data. Math. Comp. 75 1913–1929 (electronic).Mathematical Reviews (MathSciNet): MR2240642
Zentralblatt MATH: 1103.62031
Digital Object Identifier: doi:10.1090/S0025-5718-06-01873-4 - [22] Nashed, M. Z. and Wahba, G. (1974). Generalized inverses in reproducing kernel spaces: An approach to regularization of linear operator equations. SIAM J. Math. Anal. 5 974–987.Mathematical Reviews (MathSciNet): MR358405
Zentralblatt MATH: 0287.47009
Digital Object Identifier: doi:10.1137/0505095 - [23] Rudin, W. (1991). Functional Analysis, 2nd ed. McGraw-Hill, New York.Mathematical Reviews (MathSciNet): MR1157815
- [24] Ruymgaart, F. H. (1993). A unified approach to inversion problems in statistics. Math. Methods Statist. 2 130–146.
- [25] Skorohod, A. V. (1974). Integration in Hilbert Space. Ergebnisse der Mathematik und Ihrer Grenzgebiete 79. Springer, New York. Translated from the Russian by Kenneth Wickwire.Mathematical Reviews (MathSciNet): MR466482
- [26] Stuart, A. M. (2010). Inverse problems: A Bayesian perspective. Acta Numer. 19 451–559.Mathematical Reviews (MathSciNet): MR2652785
Digital Object Identifier: doi:10.1017/S0962492910000061
Zentralblatt MATH: 1242.65142 - [27] Tikhonov, A. N. and Arsenin, V. Y. (1977). Solutions of Ill-posed Problems. V. H. Winston and Sons, Washington, DC. Translated from the Russian, Preface by translation editor Fritz John, Scripta Series in Mathematics.
- [28] van der Vaart, A. (1991). On differentiable functionals. Ann. Statist. 19 178–204.Mathematical Reviews (MathSciNet): MR1091845
Zentralblatt MATH: 0732.62035
Digital Object Identifier: doi:10.1214/aos/1176347976
Project Euclid: euclid.aos/1176347976 - [29] van der Vaart, A. and van Zanten, H. (2007). Bayesian inference with rescaled Gaussian process priors. Electron. J. Stat. 1 433–448 (electronic).Mathematical Reviews (MathSciNet): MR2357712
Zentralblatt MATH: 1140.62066
Digital Object Identifier: doi:10.1214/07-EJS098
Project Euclid: euclid.ejs/1193074818 - [30] van der Vaart, A. W. (1988). Statistical Estimation in Large Parameter Spaces. CWI Tract 44. Stichting Mathematisch Centrum Centrum voor Wiskunde en Informatica, Amsterdam.
- [31] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge.
- [32] van der Vaart, A. W. and van Zanten, J. H. (2008). Reproducing kernel Hilbert spaces of Gaussian priors. In Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh. Inst. Math. Stat. Collect. 3 200–222. IMS, Beachwood, OH.Mathematical Reviews (MathSciNet): MR2459226
Digital Object Identifier: doi:10.1214/074921708000000156 - [33] van Rooij, A. C. M. and Ruymgaart, F. H. (2001). Abstract inverse estimation with application to deconvolution on locally compact abelian groups. Ann. Inst. Statist. Math. 53 781–798.Mathematical Reviews (MathSciNet): MR1880812
Zentralblatt MATH: 1087.62504
Digital Object Identifier: doi:10.1023/A:1014665305349
- You have access to this content.
- You have partial access to this content.
- You do not have access to this content.
More like this
- Frequentist coverage of adaptive nonparametric Bayesian credible sets
Szabó, Botond, van der Vaart, A. W., and van Zanten, J. H., The Annals of Statistics, 2015 - Adaptive Bayesian credible sets in regression with a Gaussian process prior
Sniekers, Suzanne and van der Vaart, Aad, Electronic Journal of Statistics, 2015 - On coverage and local radial rates of credible sets
Belitser, Eduard, The Annals of Statistics, 2017
- Frequentist coverage of adaptive nonparametric Bayesian credible sets
Szabó, Botond, van der Vaart, A. W., and van Zanten, J. H., The Annals of Statistics, 2015 - Adaptive Bayesian credible sets in regression with a Gaussian process prior
Sniekers, Suzanne and van der Vaart, Aad, Electronic Journal of Statistics, 2015 - On coverage and local radial rates of credible sets
Belitser, Eduard, The Annals of Statistics, 2017 - Rate-optimal posterior contraction for sparse PCA
Gao, Chao and Zhou, Harrison H., The Annals of Statistics, 2015 - On adaptive posterior concentration rates
Hoffmann, Marc, Rousseau, Judith, and Schmidt-Hieber, Johannes, The Annals of Statistics, 2015 - Rates of contraction of posterior distributions based on Gaussian process priors
van der Vaart, A. W. and van Zanten, J. H., The Annals of Statistics, 2008 - Bernstein–von Mises theorems for statistical inverse problems II: compound Poisson processes
Nickl, Richard and Söhl, Jakob, Electronic Journal of Statistics, 2019 - Bayesian inverse problems with non-conjugate priors
Ray, Kolyan, Electronic Journal of Statistics, 2013 - Adaptive posterior contraction rates for the horseshoe
van der Pas, Stéphanie, Szabó, Botond, and van der Vaart, Aad, Electronic Journal of Statistics, 2017 - Supremum norm posterior contraction and credible sets for nonparametric multivariate regression
Yoo, William Weimin and Ghosal, Subhashis, The Annals of Statistics, 2016