Bayesian Analysis

Nonparametric Goodness of Fit via Cross-Validation Bayes Factors

Jeffrey D. Hart and Taeryon Choi

Full-text: Open access

Abstract

A nonparametric Bayes procedure is proposed for testing the fit of a parametric model for a distribution. Alternatives to the parametric model are kernel density estimates. Data splitting makes it possible to use kernel estimates for this purpose in a Bayesian setting. A kernel estimate indexed by bandwidth is computed from one part of the data, a training set, and then used as a model for the rest of the data, a validation set. A Bayes factor is calculated from the validation set by comparing the marginal for the kernel model with the marginal for the parametric model of interest. A simulation study is used to investigate how large the training set should be, and examples involving astronomy and wind data are provided. A proof of Bayes consistency of the proposed test is also provided.

Article information

Source
Bayesian Anal., Volume 12, Number 3 (2017), 653-677.

Dates
First available in Project Euclid: 17 August 2016

Permanent link to this document
https://projecteuclid.org/euclid.ba/1471454532

Digital Object Identifier
doi:10.1214/16-BA1018

Mathematical Reviews number (MathSciNet)
MR3655871

Zentralblatt MATH identifier
1384.62147

Subjects
Primary: 62G10: Hypothesis testing 62F15: Bayesian inference
Secondary: 62G05: Estimation

Keywords
bandwidth selection Bayes factor consistency cross validation goodness-of-fit tests kernel density estimates

Rights
Creative Commons Attribution 4.0 International License.

Citation

Hart, Jeffrey D.; Choi, Taeryon. Nonparametric Goodness of Fit via Cross-Validation Bayes Factors. Bayesian Anal. 12 (2017), no. 3, 653--677. doi:10.1214/16-BA1018. https://projecteuclid.org/euclid.ba/1471454532


Export citation

References

  • Ahmad, I. and Lin, P.-E. (1976). “A nonparametric estimation of the entropy for absolutely continuous distributions.” IEEE Transactions on Information Theory, 22: 372–375.
  • Alqallaf, F. and Gustafson, P. (2001). “On cross-validation of Bayes models.” Canadian Journal of Statistics, 29: 333–340.
  • Berger, J. and Guglielmi, A. (2001). “Bayesian and conditional frequentist testing of a parametric model versus nonparametric alternatives.” Journal of the American Statistical Association, 96: 174–184.
  • Berger, J. and Pericchi, L. (1996). “The intrinsic Bayes factor for model selection and prediction.” Journal of the American Statistical Association, 91: 109–122.
  • Bickel, P. and Rosenblatt, M. (1973). “On some global measures of the deviation of density function estimates.” Annals of Statistics, 1: 1071–1095.
  • Ciardullo, R., Feldmeier, J. J., Jacoby, G. H., de Naray, R. K., Laychak, M. B., and Durrell, P. R. (2002). “Planetary Nebulae as Standard Candles. XII. Connecting the Population I and Population II Distance Scales.” Astrophysical Journal, 577: 31–50.
  • Ciardullo, R., Jacoby, G. H., Ford, H. C., and Neil, J. D. (1989). “Planetary nebulae as standard candles. II. The calibration in M31 and its companions.” Astrophysical Journal, 339: 53–69.
  • Ferguson, T. (1983). “Bayesian density estimation by mixtures of normal distributions.” In: Rizvi, H. and Rustagi, J. (eds.), Recent Advances in Statistics, 287–302. New York: Academic Press.
  • Hall, P. and Marron, J. (1987). “Extent to which least-squares cross-validation minimises integrated squared error in nonparametric density estimation.” Probability Theory and Related Fields, 74: 567–581.
  • Hoeffding, W. (1963). “Probability inequalities for sums of bounded random variables.” Journal of the American Statistical Association, 58: 13–30.
  • Jeffreys, H. (1961). Theory of Probability. Third edition. Clarendon Press, Oxford.
  • Johnson, V. and Rossell, D. (2010). “On the use of non-local prior densities in Bayesian hypothesis tests.” Journal of the Royal Statistical Society B, 72: 143–170.
  • Mardia, K. V. and Jupp, P. E. (2000). Directional Statistics. Wiley Series in Probability and Statistics. Chichester: Wiley.
  • McVinish, R., Rousseau, J., and Mengersen, K. (2009). “Bayesian goodness of fit testing with mixtures of triangular distributions.” Scandinavian Journal of Statistics, 36: 337–354.
  • Müller, P. and Quintana, F. (2004). “Nonparametric Bayesian data analysis.” Statistical Science, 19: 95–110.
  • Rayner, J., Thas, O., and Best, D. (2009). Smooth Tests of Goodness of Fit: Using R. Wiley Series in Probability and Statistics. Chichester, UK: John Wiley & Sons, Ltd, second edition.
  • Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Monographs on Statistics and Applied Probability. London: Chapman & Hall.
  • Tokdar, S. and Martin, R. (2013). “Bayesian test of normality versus a Dirichlet process mixture alternative.” Unpublished manuscript, arXiv:1108.2883v3 [math.ST].
  • Tokdar, S. T., Chakrabarti, A., and Ghosh, J. K. (2010). “Bayesian nonparametric goodness of fit tests.” In: Chen, M.-H., Dey, D. K., Müller, P., Sun, D., and Ye, K. (eds.), Frontiers of statistical decision making and Bayesian analysis. New York: Springer. In honor of James O. Berger.
  • van der Laan, M., Dudoit, S., and Keleş, S. (2004). “Asymptotic optimality of likelihood-based cross-validation.” Statistical Applications in Genetics and Molecular Biology, 3: online publication.
  • Xu, X., Lu, P., MacEachern, S., and Xu, R. (2011). “Calibrated Bayes factors for model comparison.” Unpublished manuscript.