Bayesian Analysis

Low Information Omnibus (LIO) Priors for Dirichlet Process Mixture Models

Yushu Shi, Michael Martens, Anjishnu Banerjee, and Purushottam Laud

Full-text: Open access

Abstract

Dirichlet process mixture (DPM) models provide flexible modeling for distributions of data as an infinite mixture of distributions from a chosen collection. Specifying priors for these models in individual data contexts can be challenging. In this paper, we introduce a scheme which requires the investigator to specify only simple scaling information. This is used to transform the data to a fixed scale on which a low information prior is constructed. Samples from the posterior with the rescaled data are transformed back for inference on the original scale. The low information prior is selected to provide a wide variety of components for the DPM to generate flexible distributions for the data on the fixed scale. The method can be applied to all DPM models with kernel functions closed under a suitable scaling transformation. Construction of the low information prior, however, is kernel dependent. Using DPM-of-Gaussians and DPM-of-Weibulls models as examples, we show that the method provides accurate estimates of a diverse collection of distributions that includes skewed, multimodal, and highly dispersed members. With the recommended priors, repeated data simulations show performance comparable to that of standard empirical estimates. Finally, we show weak convergence of posteriors with the proposed priors for both kernels considered.

Article information

Source
Bayesian Anal., Volume 14, Number 3 (2019), 677-702.

Dates
First available in Project Euclid: 11 June 2019

Permanent link to this document
https://projecteuclid.org/euclid.ba/1560240023

Digital Object Identifier
doi:10.1214/18-BA1119

Mathematical Reviews number (MathSciNet)
MR3960766

Zentralblatt MATH identifier
07089622

Keywords
Bayesian nonparametric methods density estimation survival analysis low-information prior Dirichlet process mixture model

Rights
Creative Commons Attribution 4.0 International License.

Citation

Shi, Yushu; Martens, Michael; Banerjee, Anjishnu; Laud, Purushottam. Low Information Omnibus (LIO) Priors for Dirichlet Process Mixture Models. Bayesian Anal. 14 (2019), no. 3, 677--702. doi:10.1214/18-BA1119. https://projecteuclid.org/euclid.ba/1560240023


Export citation

References

  • Antoniak, C. E. (1974). “Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems.” The annals of statistics, 1152–1174.
  • Argiento, R., Bianchini, I., and Guglielmi, A. (2016). “A blocked Gibbs sampler for NGG-mixture models via a priori truncation.” Statistics and Computing, 26(3): 641–661.
  • Argiento, R., Guglielmi, A., and Pievatolo, A. (2010). “Bayesian density estimation and model selection using nonparametric hierarchical mixtures.” Computational Statistics & Data Analysis, 54(4): 816–832.
  • Canale, A. and Prünster, I. (2017). “Robustifying Bayesian nonparametric mixtures for count data.” Biometrics, 73(1): 174–184.
  • Chen, X. (2007). “A new generalization of Chebyshev inequality for random vectors.” arXiv preprint arXiv:0707.0805.
  • Choi, T. and Schervish, M. J. (2007). “On posterior consistency in nonparametric regression problems.” Journal of Multivariate Analysis, 98(10): 1969–1987.
  • De Iorio, M., Müller, P., Rosner, G. L., and MacEachern, S. N. (2004). “An ANOVA model for dependent random measures.” Journal of the American Statistical Association, 99(465): 205–215.
  • Dorazio, R. M., Mukherjee, B., Zhang, L., Ghosh, M., Jelks, H. L., and Jordan, F. (2008). “Modeling unobserved sources of heterogeneity in animal abundance using a Dirichlet process prior.” Biometrics, 64(2): 635–644.
  • Escobar, M. D. and West, M. (1995). “Bayesian density estimation and inference using mixtures.” Journal of the American Statistical Association, 90(430): 577–588.
  • Ferguson, T. S. (1973). “A Bayesian analysis of some nonparametric problems.” The Annals of Statistics, 1(2): 209–230.
  • Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2014). Bayesian data analysis 3. Chapman and Hall/CRC.
  • Gelman, A., Jakulin, A., Pittau, M. G., and Su, Y.-S. (2008). “A weakly informative default prior distribution for logistic and other regression models.” The Annals of Applied Statistics, 2(4): 1360–1383.
  • Ghosal, S., Ghosh, J. K., and Ramamoorthi, R. (1999). “Posterior consistency of Dirichlet mixtures in density estimation.” The Annals of Statistics, 27(1): 143–158.
  • James, L. F., Lijoi, A., and Prünster, I. (2009). “Posterior analysis for normalized random measures with independent increments.” Scandinavian Journal of Statistics, 36(1): 76–97.
  • Jara, A., Hanson, T. E., Quintana, F. A., Müller, P., and Rosner, G. L. (2011). “DPpackage: Bayesian semi-and nonparametric modeling in R.” Journal of Statistical Software, 40(5): 1–30.
  • Kottas, A. (2006). “Nonparametric Bayesian survival analysis using mixtures of Weibull distributions.” Journal of Statistical Planning and Inference, 136(3): 578–596.
  • Lijoi, A., Mena, R. H., and Prünster, I. (2005). “Hierarchical mixture modeling with normalized inverse-Gaussian priors.” Journal of the American Statistical Association, 100(472): 1278–1291.
  • Lijoi, A., Mena, R. H., and Prünster, I. (2007). “Controlling the reinforcement in Bayesian non-parametric mixture models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(4): 715–740.
  • Lijoi, A., Prünster, I., and Walker, S. G. (2008). “Investigating nonparametric priors with Gibbs structure.” Statistica Sinica, 1653–1668.
  • Lo, A. Y. (1984). “On a class of Bayesian nonparametric estimates: I. Density estimates.” The Annals of Statistics, 12(1): 351–357.
  • Neal, R. M. (2000). “Markov chain sampling methods for Dirichlet process mixture models.” Journal of Computational and Graphical Statistics, 9(2): 249–265.
  • Regazzini, E., Lijoi, A., and Prünster, I. (2003). “Distributional results for means of normalized random measures with independent increments.” Annals of Statistics, 560–585.
  • Sethuraman, J. (1994). “A constructive definition of Dirichlet priors.” Statistica Sinica, 4(2): 639 – 650.
  • Y. Shi, M. Martens, A. Banerjee, and P. Laud (2018). “Supplementary Material for “Low Information Omnibus (LIO) Priors for Dirichlet Process Mixture Models”.” Bayesian Analysis.
  • Tokdar, S. T. (2006). “Posterior consistency of Dirichlet location-scale mixture of normals in density estimation and regression.” Sankhya: The Indian Journal of Statistics, 68(1): 90–110.
  • Turnbull, B. W. (1974). “Nonparametric estimation of a survivorship function with doubly censored data.” Journal of the American Statistical Association, 69(345): 169–173.
  • Walker, S. (2004). “New approaches to Bayesian consistency.” Annals of Statistics, 32(5): 2028–2043.
  • Wu, Y. and Ghosal, S. (2008). “Kullback Leibler property of kernel mixture priors in Bayesian density estimation.” Electronic Journal of Statistics, 2: 298–331.
  • Wu, Y. and Ghosal, S. (2010). “The L1-consistency of Dirichlet mixtures in multivariate Bayesian density estimation.” Journal of Multivariate Analysis, 101(10): 2411–2419.

Supplemental materials