Bernoulli

  • Bernoulli
  • Volume 25, Number 2 (2019), 1256-1288.

Truncated random measures

Trevor Campbell, Jonathan H. Huggins, Jonathan P. How, and Tamara Broderick

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

Completely random measures (CRMs) and their normalizations are a rich source of Bayesian nonparametric priors. Examples include the beta, gamma, and Dirichlet processes. In this paper, we detail two major classes of sequential CRM representations—series representations and superposition representations—within which we organize both novel and existing sequential representations that can be used for simulation and posterior inference. These two classes and their constituent representations subsume existing ones that have previously been developed in an ad hoc manner for specific processes. Since a complete infinite-dimensional CRM cannot be used explicitly for computation, sequential representations are often truncated for tractability. We provide truncation error analyses for each type of sequential representation, as well as their normalized versions, thereby generalizing and improving upon existing truncation error bounds in the literature. We analyze the computational complexity of the sequential representations, which in conjunction with our error bounds allows us to directly compare representations and discuss their relative efficiency. We include numerous applications of our theoretical results to commonly-used (normalized) CRMs, demonstrating that our results enable a straightforward representation and analysis of CRMs that has not previously been available in a Bayesian nonparametric context.

Article information

Source
Bernoulli, Volume 25, Number 2 (2019), 1256-1288.

Dates
Received: February 2017
Revised: July 2017
First available in Project Euclid: 6 March 2019

Permanent link to this document
https://projecteuclid.org/euclid.bj/1551862850

Digital Object Identifier
doi:10.3150/18-BEJ1020

Mathematical Reviews number (MathSciNet)
MR3920372

Zentralblatt MATH identifier
07049406

Keywords
Bayesian nonparametrics completely random measure normalized completely random measure Poisson point process truncation

Citation

Campbell, Trevor; Huggins, Jonathan H.; How, Jonathan P.; Broderick, Tamara. Truncated random measures. Bernoulli 25 (2019), no. 2, 1256--1288. doi:10.3150/18-BEJ1020. https://projecteuclid.org/euclid.bj/1551862850


Export citation

References

  • [1] Abramowitz, M. and Stegun, I.A., eds. (1964). Handbook of Mathematical Functions, with Formulas, Graphs, and Mathematical Tables. New York: Dover Publications, Inc.
  • [2] Airoldi, E.M., Blei, D., Erosheva, E.A. and Fienberg, S.E. (2014). Handbook of Mixed Membership Models and Their Applications. Boca Raton, FL: CRC Press.
  • [3] Arbel, J. and Prünster, I. (2017). A moment-matching Ferguson & Klass algorithm. Stat. Comput. 27 3–17.
  • [4] Argiento, R., Bianchini, I. and Guglielmi, A. (2016). A blocked Gibbs sampler for NGG-mixture models via a priori truncation. Stat. Comput. 26 641–661.
  • [5] Banjevic, D., Ishwaran, H. and Zarepour, M. (2002). A recursive method for functionals of Poisson processes. Bernoulli 8 295–311.
  • [6] Blei, D.M. and Jordan, M.I. (2006). Variational inference for Dirichlet process mixtures. Bayesian Anal. 1 121–143.
  • [7] Bondesson, L. (1982). On simulation from infinitely divisible distributions. Adv. in Appl. Probab. 14 855–869.
  • [8] Brix, A. (1999). Generalized gamma measures and shot-noise Cox processes. Adv. in Appl. Probab. 31 929–953.
  • [9] Broderick, T., Jordan, M.I. and Pitman, J. (2012). Beta processes, stick-breaking and power laws. Bayesian Anal. 7 439–475.
  • [10] Broderick, T., Mackey, L., Paisley, J. and Jordan, M.I. (2015). Combinatorial clustering and the beta negative binomial process. IEEE Trans. Pattern Anal. Mach. Intell. 37 290–306.
  • [11] Broderick, T., Wilson, A.C. and Jordan, M.I. (2018). Posteriors, conjugacy, and exponential families for completely random measures. Bernoulli 24 3181–3221.
  • [12] Campbell, T., Huggins, J.H., How, J.P. and Broderick, T. (2019). Supplement to “Truncated random measures”. DOI:10.3150/18-BEJ1020SUPP.
  • [13] Doshi-Velez, F., Miller, K.T., Van Gael, J. and Teh, Y.W. (2009). Variational inference for the Indian buffet process. In International Conference on Artificial Intelligence and Statistics.
  • [14] Ferguson, T.S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1 209–230.
  • [15] Ferguson, T.S. and Klass, M.J. (1972). A representation of independent increment processes without Gaussian components. Ann. Math. Stat. 43 1634–1643.
  • [16] Gelfand, A.E. and Kottas, A. (2002). A computational approach for full nonparametric Bayesian inference under Dirichlet process mixture models. J. Comput. Graph. Statist. 11 289–305.
  • [17] Gumbel, E.J. (1954). Statistical Theory of Extreme Values and Some Practical Applications. A Series of Lectures. National Bureau of Standards Applied Mathematics Series 33. Washington, DC: U.S. Government Printing Office.
  • [18] Hjort, N.L. (1990). Nonparametric Bayes estimators based on beta processes in models for life history data. Ann. Statist. 18 1259–1294.
  • [19] Ishwaran, H. and James, L.F. (2001). Gibbs sampling methods for stick-breaking priors. J. Amer. Statist. Assoc. 96 161–173.
  • [20] Ishwaran, H. and James, L.F. (2002). Approximate Dirichlet process computing in finite normal mixtures: Smoothing and prior information. J. Comput. Graph. Statist. 11 508–532.
  • [21] Ishwaran, H. and Zarepour, M. (2002). Exact and approximate sum representations for the Dirichlet process. Canad. J. Statist. 30 269–283.
  • [22] James, L.F. (2002). Poisson process partition calculus with applications to exchangeable models and Bayesian nonparametrics. Preprint. Available at arXiv:0205093.
  • [23] James, L.F. (2013). Stick-breaking PG($\alpha,\zeta$)-generalized gamma processes. Preprint. Available at arXiv:1308.6570.
  • [24] James, L.F. (2014). Poisson latent feature calculus for generalized Indian buffet processes. Preprint. Available at arXiv:1411.2936v3.
  • [25] James, L.F., Lijoi, A. and Prünster, I. (2009). Posterior analysis for normalized random measures with independent increments. Scand. J. Stat. 36 76–97.
  • [26] Kim, Y. (1999). Nonparametric Bayesian estimators for counting processes. Ann. Statist. 27 562–588.
  • [27] Kingman, J.F.C. (1967). Completely random measures. Pacific J. Math. 21 59–78.
  • [28] Kingman, J.F.C. (1993). Poisson Processes. Oxford Studies in Probability 3. New York: The Clarendon Press, Oxford Univ. Press.
  • [29] Kingman, J.F.C., Taylor, S.J., Hawkes, A.G., Walker, A.M., Cox, D.R., Smith, A.F.M., Hill, B.M., Burville, P.J. and Leonard, T. (1975). Random discrete distribution. J. Roy. Statist. Soc. Ser. B 37 1–22. With a discussion by S.J. Taylor, A.G. Hawkes, A.M. Walker, D.R. Cox, A.F.M. Smith, B.M. Hill, P.J. Burville, T. Leonard and a reply by the author.
  • [30] Lijoi, A., Mena, R.H. and Prünster, I. (2005). Bayesian nonparametric analysis for a generalized Dirichlet process prior. Stat. Inference Stoch. Process. 8 283–309.
  • [31] Lijoi, A., Mena, R.H. and Prünster, I. (2007). Controlling the reinforcement in Bayesian non-parametric mixture models. J. Roy. Statist. Soc. Ser. B 69 715–740.
  • [32] Lijoi, A. and Prünster, I. (2003). On a normalized random measure with independent increments relevant to Bayesian nonparametric inference. In Proceedings of the 13th European Young Statisticians Meeting 123–124.
  • [33] Lijoi, A. and Prünster, I. (2010). Models beyond the Dirichlet process. In Bayesian Nonparametrics (N.L. Hjort, C. Holmes, P. Müller and S. Walker, eds.). Camb. Ser. Stat. Probab. Math. 28 80–136. Cambridge: Cambridge Univ. Press.
  • [34] Maddison, C., Tarlow, D. and Minka, T.P. (2014). A∗ sampling. In Advances in Neural Information Processing Systems.
  • [35] Muliere, P. and Tardella, L. (1998). Approximating distributions of random functionals of Ferguson–Dirichlet priors. Canad. J. Statist. 26 283–297.
  • [36] Orbanz, P. (2010). Conjugate projective limits. Preprint. Available at arXiv:1012.0363.
  • [37] Paisley, J.W., Blei, D.M. and Jordan, M.I. (2012). Stick-breaking beta processes and the Poisson process. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research 22 850–858.
  • [38] Paisley, J.W., Zaas, A.K., Woods, C.W., Ginsburg, G.S. and Carin, L. (2010). A stick-breaking construction of the beta process. In International Conference on Machine Learning.
  • [39] Perman, M. (1993). Order statistics for jumps of normalised subordinators. Stochastic Process. Appl. 46 267–281.
  • [40] Perman, M., Pitman, J. and Yor, M. (1992). Size-biased sampling of Poisson point processes and excursions. Probab. Theory Related Fields 92 21–39.
  • [41] Pitman, J. (2003). Poisson–Kingman partitions. In Statistics and Science: A Festschrift for Terry Speed. Institute of Mathematical Statistics Lecture Notes—Monograph Series 40 1–34. Beachwood, OH: IMS.
  • [42] Regazzini, E., Lijoi, A. and Prünster, I. (2003). Distributional results for means of normalized random measures with independent increments. Ann. Statist. 31 560–585. Dedicated to the memory of Herbert E. Robbins.
  • [43] Rosiński, J. (1990). On series representations of infinitely divisible random vectors. Ann. Probab. 18 405–430.
  • [44] Rosiński, J. (2001). Series representations of Lévy processes from the perspective of point processes. In Lévy Processes (O. Barndorff-Nielson, S. Resnick and T. Mikosch, eds.) 401–415. Boston, MA: Birkhäuser.
  • [45] Roy, D. (2014). The continuum-of-urns scheme, generalized beta and Indian buffer processes, and hierarchies thereof. Preprint. Available at arXiv:1501.00208.
  • [46] Roychowdhury, A. and Kulis, B. (2015). Gamma processes, stick-breaking, and variational inference. In International Conference on Artificial Intelligence and Statistics.
  • [47] Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Statist. Sinica 4 639–650.
  • [48] Teh, Y.W. and Görür, D. (2009). Indian buffet processes with power-law behavior. In Advances in Neural Information Processing Systems.
  • [49] Thibaux, R. and Jordan, M.I. (2007). Hierarchical beta processes and the Indian buffet process. In International Conference on Artificial Intelligence and Statistics.
  • [50] Titsias, M. (2008). The infinite gamma-Poisson feature model. In Advances in Neural Information Processing Systems.
  • [51] Zhou, M., Hannah, L., Dunson, D. and Carin, L. (2012). Beta-negative binomial process and Poisson factor analysis. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research 22 1462–1471.

Supplemental materials

  • Supplement to “Truncated random measures”. Proofs for all results developed in this paper along with some additional example applications and simulations.