The Annals of Statistics

Bayesian goodness-of-fit testing using infinite-dimensional exponential families

Isabella Verdinelli and Larry Wasserman

Full-text: Open access

Abstract

We develop a nonparametric Bayes factor for testing the fit of a parametric model. We begin with a nominal parametric family which we then embed into an infinite-dimensional exponential family. The new model then has a parametric and nonparametric component. We give the log density of the nonparametric component a Gaussian process prior. An asymptotic consistency requirement puts a restriction on the form of the prior, leaving us with a single hyperparameter for which we suggest a default value based on simulation experience. Then we construct a Bayes factor to test the nominal model versus the semiparametric alternative. Finally, we show that the Bayes factor is consistent. The proof of the consistency is based on approximating the model by a sequence of exponential families.

Article information

Source
Ann. Statist., Volume 26, Number 4 (1998), 1215-1241.

Dates
First available in Project Euclid: 21 June 2002

Permanent link to this document
https://projecteuclid.org/euclid.aos/1024691240

Digital Object Identifier
doi:10.1214/aos/1024691240

Mathematical Reviews number (MathSciNet)
MR1647645

Zentralblatt MATH identifier
0930.62027

Subjects
Primary: 62F15 62G10

Keywords
Bayes factor consistency Gaussian process prior Markov chain Monte Carlo nonparametric Bayesian inference sieve

Citation

Verdinelli, Isabella; Wasserman, Larry. Bayesian goodness-of-fit testing using infinite-dimensional exponential families. Ann. Statist. 26 (1998), no. 4, 1215--1241. doi:10.1214/aos/1024691240. https://projecteuclid.org/euclid.aos/1024691240


Export citation

References

  • ANDERSON, T. W. and DAS GUPTA, S. 1963. Some inequalities on characteristic roots of matrices. Biometrika 50 522 524.Z.
  • ANDREWS, D. F. and HERZERG, A. 1985. Data: A Collection of Problems From Many Fields for the Student and Research Worker. Springer, New York. Z.
  • ANTONIAK, C. E. 1974. Mixtures of Dirichlet processes with applications to non-parametric problems. Ann. Statist. 2 1152 1174. Z.
  • BARRON, A. R. 1988. The exponential convergence of posterior probabilities with implications for Bay es estimators of density functions. Technical Report 7, Dept. Statistics, Univ. Illinois. Z.
  • BARRON, A. R. and COVER, T. M. 1991. Minimum complexity density estimation. IEEE Trans. Inform. Theory 37 1034 1054. Z.
  • BARRON, A. R. and SHEU, C. H. 1991. Approximation of density functions by sequences of
  • BAy ARRI, M. J. 1985. A Bayesian test for goodness-of-fit. Unpublished manuscript. Z.
  • BERGER, J. O. and BERNARDO, J. M. 1989. Estimating a product of means: Bayesian analysis with reference priors. J. Amer. Statist. Assoc. 84 200 207. Z.
  • BERGER, J. O. and PERICCHI, L. 1996. The intrinsic Bay es factor for model selection and prediction. J. Amer. Statist. Assoc. 91 109 122. Z.
  • BOX, G. E. P. 1980. Sampling and Bay es inference in scientific modeling and robustness. J. Roy. Statist. Soc. Ser. A 143 383 430. Z.
  • BOX, G. E. P. and TIAO, G. C. 1973. Bayesian Inference in Statistical Analy sis. Addison-Wesley, Menlo Park, NY. Z.
  • BRUNK, H. D. 1978. Univariate density estimation by orthogonal series. Biometrika 65 521 528. Z.
  • CRAIN, B. 1974. Estimation of distributions using orthogonal expansions. Ann. Statist. 2 454 463. Z.
  • CRAIN, B. 1976. More on the estimation of distributions using orthogonal expansions. J. Amer. Statist. Assoc. 71 741 745.Z.
  • DELAMPADY, M. and BERGER, J. O. 1990. Lower bounds on Bay es factors for multinomial distributions, with application to chi-squared tests of fit. Ann. Statist. 18 1295 1316.
  • DIACONIS, P. and FRIEDMAN, D. 1986. On the consistency of Bay es estimates. Ann. Statist. 14 1 67. Z.
  • DICICCIO, T., KASS, R. E., RAFTERY, A. and WASSERMAN, L. 1995. Simulation methods for computing Bay es factors. Technical report, Dept. Statistics, Carnegie Mellon Univ. Z.
  • DIEBOLT, J. and ROBERT, C. 1994. Estimation of finite mixture distributions through Bayesian sampling. J. Roy. Statist. Soc. Ser. B 56 363 375. Z.
  • DUMBGEN, L. 1998. New goodness of fit tests and their application to nonparametric confidence ¨ sets. Ann. Statist. 26 288 314. Z.
  • EFRON, B. and TIBSHIRANI, R. 1996. Specially designed exponential families for density estimation. Ann. Statist. 24 2431 2461. Z.
  • EUBANK, R. and LARICCIA, V. 1992. Asy mptotic comparison Cramer Von Mises and nonpara´ metric function estimation techniques for testing goodness of fit. Ann. Statist. 20 2071 2086. Z.
  • EVANS, M. and SWARTZ, T. 1994. Distribution theory and inference for poly nomial-normal densities. Comm. Statist. Theory Methods 23 1123 1148. Z.
  • FERGUSON, T. 1973. A Bayesian analysis of some nonparametric problems. Ann. Statist. 1 209 230. Z.
  • GELFAND, A. and DEY, D. 1994. Bayesian model choice: asy mptotics and exact calculations. J. Roy. Statist. Soc. Ser. B 56 501 515. Z.
  • GELFAND, A., DEY, D. and CHANG, H. 1992. Model determination using predictive distributions Z. with implementation via sampling-based methods with discussion. In Bayesian Z. Statistics 4 J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds. 147 167. Oxford Univ. Press. Z.
  • GELFAND, A. and SMITH, A. F. M. 1990. Sampling based approaches to calculating marginal densities. J. Amer. Statist. Assoc. 85 398 409. Z.
  • GELMAN, A., MENG, X. and STERN, H. 1996. Posterior predictive assessment of model fitness via realized discrepancies. Statist. Sinica 6 733 807. Z.
  • GEWEKE, J. 1989. Modelling with normal poly nomial expansions. In Economic Complexity. Z. Chaos, Sunspots and Nonlinearity W. A. Bennett, J. Geweke and K. Shell, eds. Cambridge Univ. Press. Z. Z.
  • GOOD, I. J. 1967. A Bayesian significance test for multinomial distributions with discussion, J. Roy. Statist. Soc. Ser. B 29 399 431. Z.
  • GOOD, I. J. 1992. The Bay es non-Bay es compromise: a brief review. J. Amer. Statist. Assoc. 87 597 606. Z.
  • GRENANDER, U. 1981. Abstract Inference. Wiley, New York. Z.
  • HARDLE, W. and MAMMEN, E. 1993. Comparing nonparametric versus parametric regression ¨ fits. Ann. Statist. 21 1926 1947. Z.
  • HART, J. 1997. Nonparametric Smoothing and Lack of Fit Tests. Springer, New York. Z.
  • HJORT, N. L. 1996. Bayesian approaches to nonand semiparametric density estimation. In Z Bayesian Statistics 5 J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith,. eds. 223 253. Oxford Univ. Press. Z.
  • HJORT, N. L. and GLAD, I. 1995. Nonparametric estimation with a parametric start. Ann. Statist. 23 882 904. Z.
  • HJORT, N. L. and JONES, M. C. 1996. Locally parametric nonparametric density estimation. Ann. Statist. 24 1619 1647. Z.
  • INGLOT, T. and LEDWINA, T. 1996. Asy mptotic optimality of data-driven Ney man tests for uniformity. Ann. Statist. 24 1982 2019. Z.
  • JEFFREy S, H. 1961. Theory of Probability, 3rd ed. Clarendon Press, Oxford. Z.
  • KASS, R. E. and RAFTERY, A. 1995. Bay es factors. J. Amer. Statist. Assoc. 90 773 793. Z.
  • LAVINE, M. 1994. More aspects of Poly a tree distributions for statistical modelling. Ann. Statist. 22 1161 1176. Z.
  • LEDWINA, T. 1994. Data-driven version of Ney man's smooth test of fit. J. Amer. Statist. Assoc. 89 1000 1005. Z.
  • LENK, P. J. 1988. The logistic normal distribution for Bayesian, nonparametric, predictive densities. J. Amer. Statist. Assoc. 83 509 516.
  • LENK, P. J. 1991. Towards a practicable Bayesian nonparametric density estimator. Biometrika 78 531 543. Z.
  • LEONARD, T. 1978. Density estimation, stochastic processes, and prior information. J. Roy. Statist. Soc. Ser. B 40 113 146. Z.
  • MARRON, J. S. and WAND, M. P. 1992. Exact mean integrated squared error. Ann. Statist. 20 712 736. Z.
  • MENG, X. L. 1994. Posterior predictive p-values. Ann. Statist. 22 1142 1160. Z.
  • MENG, X. and WONG, W. 1996. Simulating ratios of normalizing constants via a simple identity: a theoretical exploration. Statist. Sinica 6 831 860. Z.
  • NEy MAN, J. 1937. ``Smooth'' test for goodness of fit. Skand. Aktvarietidskr. 20 150 199. Z.
  • NOBILE, A. 1994. Bayesian analysis of finite mixture distributions. Ph.D. dissertation, Dept. Statistics, Carnegie Mellon Univ. Z. Z. O'HAGAN, A. 1995. Fractional Bay es factors for model comparison with discussion. J. Roy. Statist. Soc. Ser. B 57 99 138. Z.
  • PORTNOY, S. 1988. Asy mptotic behavior of likelihood methods for exponential families when the number of parameter tends to infinity. Ann. Statist. 16 356 366. Z.
  • RAFTERY, A. 1992. Discussion of, ``Model determination using predictive distributions with implementation via sampling-based methods'' by A. Gelfand, D. Dey and H. Chang. In Z Bayesian Statistics 4 J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith,. eds. 160 163. Oxford Univ. Press. Z.
  • RAy NER, J. C. W. and BEST, D. J. 1990. Smooth tests of goodness of fit: an overview. Internat. Statist. Rev. 58 9 17.Z.
  • ROEDER, K. and WASSERMAN, L. 1995. Practical Bayesian density estimation using mixtures of normals. Technical Report 633, Dept. Statistics, Carnegie Mellon Univ. Z.
  • RUBIN, D. B. 1984. Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Statist. 12 1151 1172. Z.
  • SCHWARZ, G. 1978. Estimating the dimension of a model. Ann. Statist. 6 461 464. Z.
  • SHEN, X. 1995. On the properties of Bay es procedures in general parameter spaces. Technical report, Dept. Statistics, Ohio State Univ. Z.
  • SHUN, Z. and MCCULLAGH, P. 1995. Laplace approximation of high dimensional integrals. J. Roy. Statist. Soc. B 57 749 760. Z.
  • SILVERMAN, B. W. 1986. Density Estimation for Statistics and Data Analy sis. Chapman and Hall, New York. Z.
  • TANNER, M. and WONG, W. 1987. The calculation of posterior distributions by data augmentation. J. Amer. Statist. Assoc. 82 528 540. Z.
  • TARTER, M. and LOCK, M. 1993. Model-Free Curve Estimation. Chapman and Hall, New York. Z.
  • THORBURN, D. 1986. A Bayesian approach to density estimation. Biometrika 73 65 76. Z.
  • TIERNEY, L. 1994. Markov chains for exploring posterior distributions. Ann. Statist. 22 1701 1762. Z.
  • VERDINELLI, I. and WASSERMAN, L. 1995. Computing Bay es factors by using a generalization of the Savage Dickey density ratio. J. Amer. Statist. Assoc. 90 614 618. Z.
  • WAHBA, G. 1981. Data-based optimal smoothing of orthogonal series density estimates. Ann. Statist. 9 146 156. Z. Z
  • WEST, M. 1992. Modelling with mixtures. In Bayesian Statistics 4 J. M. Bernardo, J. O.. Berger, A. P. Dawid and A. F. M. Smith, eds. 503 524. Oxford Univ. Press. Z.
  • WHITTLE, P. 1958. On the smoothing of probability density functions. J. Roy. Statist. Soc. Ser. B 20 334 343.
  • 00185 ROMA, PITTSBURGH, PENNSy LVANIA 15213-3890 ITALY E-MAIL: larry@stat.cmu.edu