The Annals of Statistics

Asymptotic normality with small relative errors of posterior probabilities of half-spaces

R. M. Dudley and D. Haughton

Full-text: Open access

Abstract

Let $\Theta$ be a parameter space included in a finite-dimensional Euclidean space and let $A$ be a half-space. Suppose that the maximum likelihood estimate $\theta_n$ of $\theta$ is not in $A$ (otherwise, replace $A$ by its complement) and let $\Delta$ be the maximum log likelihood (at $\theta_n$) minus the maximum log likelihood over the boundary $\partial A$. It is shown that under some conditions, uniformly over all half-spaces $A$, either the posterior probability of $A$ is asymptotic to $\Phi(-\sqrt{2\Delta}\,)$ where $\Phi$ is the standard normal distribution function, or both the posterior probability and its approximant go to 0 exponentially in $n$. Sharper approximations depending on the prior are also defined.

Article information

Source
Ann. Statist., Volume 30, Number 5 (2002), 1311-1344.

Dates
First available in Project Euclid: 28 October 2002

Permanent link to this document
https://projecteuclid.org/euclid.aos/1035844978

Digital Object Identifier
doi:10.1214/aos/1035844978

Mathematical Reviews number (MathSciNet)
MR1936321

Zentralblatt MATH identifier
1014.62031

Subjects
Primary: 62F15: Bayesian inference
Secondary: 60F99: None of the above, but in this section 62F05: Asymptotic properties of tests

Keywords
Bernstein-von Mises theorem gamma tail probabilities intermediate deviations Jeffreys prior Mills' ratio

Citation

Dudley, R. M.; Haughton, D. Asymptotic normality with small relative errors of posterior probabilities of half-spaces. Ann. Statist. 30 (2002), no. 5, 1311--1344. doi:10.1214/aos/1035844978. https://projecteuclid.org/euclid.aos/1035844978


Export citation

References

  • ALON, N., BEN-DAVID, S., CESA-BIANCHI, N. and HAUSSLER, D. (1997). Scale-sensitive dimensions, uniform convergence, and learnability. J. ACM 44 615-631.
  • BARNDORFF-NIELSEN, O. E. and WOOD, A. T. A. (1998). On large deviations and choice of ancillary for p and r. Bernoulli 4 35-63.
  • BERGER, J. O. and MORTERA, J. (1999). Default Bay es factors for nonnested hy pothesis testing. J. Amer. Statist. Assoc. 94 542-554.
  • BERK, R. H. (1966). Limiting behavior of posterior distributions when the model is incorrect. Ann. Math. Statist. 37 51-58. [Correction (1996) 37 745-746.]
  • BICKEL, P. J. and GHOSH, J. K. (1990). A decomposition for the likelihood ratio statistic and the Bartlett correction-a Bayesian argument. Ann. Statist. 18 1070-1090.
  • BLEISTEIN, N. (1966). Uniform asy mptotic expansions of integrals with stationary point near algebraic singularity. Comm. Pure Appl. Math. 19 353-370.
  • CHEN, C.-F. (1985). On asy mptotic normality of limiting density functions with Bayesian implications. J. Roy. Statist. Soc. Ser. B 47 540-546.
  • CHOY, S. T. B. and SMITH, A. F. M. (1997). On robust analysis of a normal location parameter. J. Roy. Statist. Soc. Ser. B 59 463-474.
  • DANIELS, H. E. (1987). Tail probability approximations. Internat. Statist. Rev. 55 37-48.
  • DICICCIO, T. J. and MARTIN, M. A. (1991). Approximations of marginal tail probabilities for a class of smooth functions with applications to Bayesian and conditional inference. Biometrika 78 891-902.
  • DICICCIO, T. J. and STERN, S. E. (1993). On Bartlett adjustments for approximate Bayesian inference. Biometrika 80 731-740.
  • DUDLEY, R. M. (1993). Real Analy sis and Probability, 2nd ed., corrected. Chapman and Hall, New York.
  • DUDLEY, R. M. (1998). Consistency of M-estimators and one-sided bracketing. In High Dimensional Probability (E. Eberlein, M. Hahn and M. Talagrand, eds.) 33-58. Birkhäuser, Basel.
  • DUDLEY, R. M., GINÉ, E. and ZINN, J. (1991). Uniform and universal Glivenko-Cantelli classes. J. Theoret. Probab. 4 485-510.
  • DUDLEY, R. M. and HAUGHTON, D. (1997). Information criteria for multiple data sets and restricted parameters. Statist. Sinica 7 265-284.
  • DUDLEY, R. M. and HAUGHTON, D. (2001). One-sided hy potheses in a multinomial model. In Goodness-of-Fit Tests and Model Validity (C. Huber-Carol, N. Balakrishnan, M. S. Nikulin and M. Mesbah, eds.) 387-399. Birkhäuser, Boston.
  • ERKANLI, A. (1994). Laplace approximations for posterior expectations when the mode occurs at the boundary of the parameter space. J. Amer. Statist. Assoc. 89 250-258.
  • FRASER, D. A. S., REID, N. and WU, J. (1999). A simple general formula for tail probabilities for frequentist and Bayesian inference. Biometrika 86 249-264.
  • FULKS, W. and SATHER, J. O. (1961). Asy mptotics. II. Laplace's method for multiple integrals. Pacific J. Math. 11 185-192.
  • HAUGHTON, D. (1984). On the choice of a model to fit data from an exponential family. Ph.D. dissertation, MIT.
  • HAUGHTON, D. M. A. (1988). On the choice of a model to fit data from an exponential family. Ann. Statist. 16 342-355.
  • HIPP C. and MICHEL, R. (1976). On the Bernstein-v. Mises approximation of posterior distributions. Ann. Statist. 4 972-980.
  • HOLT, R. J. (1986). Computation of gamma and beta tail probabilities. Technical report, Dept. Mathematics, MIT.
  • HSU, L. C. (1948). A theorem on the asy mptotic behavior of a multiple integral. Duke Math. J. 15 623-632.
  • HUBER, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. Proc. Fifth Berkeley Sy mp. Math. Statist. Probab. 1 221-233. Univ. California Press, Berkeley.
  • JENSEN, J. L. (1995). Saddlepoint Approximations. Oxford Univ. Press.
  • JOHNSON, R. A. (1970). Asy mptotic expansions associated with posterior distributions. Ann. Math. Statist. 41 851-864.
  • LAWLEY, D. N. (1956). A general method for approximating to the distribution of likelihood ratio criteria. Biometrika 43 295-303.
  • LE CAM, L. (1953). On some asy mptotic properties of maximum likelihood estimates and related Bay es' estimates. Univ. California Publ. Statist. 1 227-329.
  • LEONARD, T. (1982). Comment. J. Amer. Statist. Assoc. 77 657-658.
  • LUGANNANI, R. and RICE, S. (1980). Saddle point approximation for the distribution of the sum of independent random variables. Adv. in Appl. Probab. 12 475-490.
  • PAULER, D. K., WAKEFIELD, J. C. and KASS, R. E. (1999). Bay es factors and approximations for variance component models. J. Amer. Statist. Assoc. 94 1242-1253.
  • PERICCHI, L. R. and SMITH, A. F. M. (1992). Exact and approximate posterior moments for a normal location parameter. J. Roy. Statist. Soc. Ser. B 54 793-804.
  • POSKITT, D. S. (1987). Precision, complexity and Bayesian model determination. J. Roy. Statist. Soc. Ser. B 49 199-208.
  • REID, N. (1996). Likelihood and higher-order approximations to tail areas: A review and annotated bibliography. Canad. J. Statist. 24 141-166.
  • RUDIN, W. (1976). Principles of Mathematical Analy sis, 3rd ed. McGraw-Hill, New York.
  • SCHWARZ, G. (1978). Estimating the dimension of a model. Ann. Statist 6 461-464.
  • SHUN, Z. and MCCULLAGH, P. (1995). Laplace approximation of high-dimensional integrals. J. Roy. Statist. Soc. Ser. B 57 749-760.
  • SKINNER, L. A. (1980). Note on the asy mptotic behavior of multidimensional Laplace integrals. SIAM J. Math. Anal. 11 911-917.
  • TALAGRAND, M. (1987). The Glivenko-Cantelli problem. Ann. Probab. 15 837-870.
  • TEMME, N. M. (1982). The uniform asy mptotic expansion of a class of integrals related to cumulative distribution functions. SIAM J. Math. Anal. 13 239-253.
  • TEMME, N. M. (1987). Incomplete Laplace integrals: Uniform asy mptotic expansion with application to the incomplete beta function. SIAM J. Math. Anal. 18 1638-1663.
  • TIERNEY, L. and KADANE, J. B. (1986). Accurate approximations for posterior moments and marginal densities. J. Amer. Statist. Assoc. 81 82-86.
  • TIERNEY, L., KASS, R. E. and KADANE, J. B. (1989). Approximate marginal densities of nonlinear functions. Biometrika 76 425-433. [Correction (1991) 78 233-234.]
  • VAN DER VAART, A. W. and WELLNER, J. A. (2000). Preservation theorems for Glivenko-Cantelli and uniform Glivenko-Cantelli classes. In High Dimensional Probability II (E. Giné, D. M. Mason and J. A. Wellner, eds.) 115-133. Birkhäuser, Boston.
  • WALKER, A. M. (1969). On the asy mptotic behaviour of posterior distributions. J. Roy. Statist. Soc. Ser. B 31 80-88.
  • WALL, H. S. (1948). Analy tic Theory of Continued Fractions. Van Nostrand, New York.
  • WONG, R. (1973). On uniform asy mptotic expansion of definite integrals. J. Approx. Theory 7 76-86.
  • WONG, R. (1989). Asy mptotic Approximations of Integrals. Academic Press, New York.
  • WOODROOFE, M. (1992). Integrable expansions for posterior distributions for one-parameter exponential families. Statist. Sinica 2 91-111.
  • CAMBRIDGE, MASSACHUSETTS 02139 E-MAIL: rmd@math.mit.edu DEPARTMENT OF MATHEMATICAL SCIENCES BENTLEY COLLEGE 175 FOREST ST.
  • WALTHAM, MASSACHUSETTS 02154 E-MAIL: dhaughton@bentley.edu