Bayesian Analysis

Bayesian dynamic density estimation

Abel Rodriguez and Enrique ter Horst

Full-text: Open access


Empirical distributions in finance and economics might show heavy tails, volatility clustering, varying mean returns and multimodality as part of their features. However, most statistical models available in the literature assume some kind of parametric form (clearly neglecting important characteristics of the data) or focus on modeling extreme events (therefore, providing no information about the rest of the distribution). In this paper we develop a Bayesian nonparametric prior for a collection of distributions evolving in discrete time. The prior is constructed by defining the distribution at any time point as a Dirichlet process mixture of Gaussian distributions, and inducing dependence through the atoms of their stick-breaking decomposition. A general construction, which allows for trends, periodicities and regressors is described. The resulting model is applied to the estimation of the time-varying travel expense distribution of employees from a major development bank comparable to the IDB, IMF and World Bank.

Article information

Bayesian Anal., Volume 3, Number 2 (2008), 339-365.

First available in Project Euclid: 22 June 2012

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Dependent Dirichlet process Nonparametric Bayes Random probability measure Travel Costs Insurance Claim Distributions


Rodriguez, Abel; ter Horst, Enrique. Bayesian dynamic density estimation. Bayesian Anal. 3 (2008), no. 2, 339--365. doi:10.1214/08-BA313.

Export citation


  • Aguilar, O., Huerta, G., Prado, R., and West, M. (1999). "Bayesian inference on latent structure in time series." In Bayesian Statistics 6, 3–26. Oxford University Press.
  • Antoniak, C. (1974). "Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems." Annals of Statistics, 2: 1152–1174.
  • Bigelow, J. and Dunson, D. (2005). "Semiparametric classification in hierarchical functional data analysis." Institute of Statistics and Decision Sciences, Duke University, Discussion paper 05-18.
  • Blackwell, D. and MacQueen, J. (1973). "Ferguson Distribution via Pólya Urn Schemes." The Annals of Statistics, 1: 353–355.
  • Box, G. and Jenkins, G. (1974). Time Series Analysis: Forecasting and Control. San Francisco: Holden-Day, 2nd edition.
  • Bush, C. and MacEachern, S. (1996). "A Semiparametric Bayesian Model for Randomised Block Designs." Biometrika, 83: 275–285.
  • Caron, F., Davy, M., Doucet, A., and Duflos, E. (2008). "Bayesian Inference for Linear Dynamic Models with Dirichlet Process Mixtures." IEEE Transaction on Signal Processing, to appear.
  • Carter, C. K. and Kohn, R. (1994). "On Gibbs sampling for state space models." Biometrika, 81: 541–553.
  • Chaveesuk, R., Srivaree-Ratana, C., and Smith, A. E. (1999). "Alternative neural network approaches to corporate bond rating." Journal of Engineering Valuation and Cost Analysis, 2: 117–131.
  • Chib, S. and Hamilton, B. (2002). "Semiparametric Bayes Analysis of Longitudinal Data Treatment Models." Journal of Econometrics, 110: 67–89.
  • DeIorio, M., Müller, P., Rosner, G., and MacEachern, S. (2004). "An ANOVA" Model for Dependent Random Measures. Journal of the American Statistical Association, 205–215.
  • Dunson, D. (2005). "Bayesian semiparametric isotonic regression for count data." Journal of the American Statistical Association, 100: 618–627.
  • –- (2006). "Bayesian dynamic modeling of latent trait distributions." Biostatistics, 7: 551–568.
  • Dunson, D., Herring, A., and Mulherin-Engel, S. (2007). "Bayesian selection and clustering of polymorphisms in functionally-related genes." Journal of the American Statistical Association.
  • Dunson, D. and Park, J.-H. (2008). "Kernel stick breaking processes." Biometrika, to appear.
  • Dunson, D., Pillai, N., and Park, J.-H. (2007). "Bayesian density regression." Journal of the Royal Statistical Society - Series B, 69: 163–183.
  • Engle, R. (1982). "Evaluation and comparison of EEG traces: Latent Structure in Non-Stationary Time Series." Econometrica, 50: 987–1008.
  • Escobar, M. (1994). "Estimating Normal Means with a Dirichlet Process Prior." Journal of the American Statistical Association, 89: 268–277.
  • Escobar, M. and West, M. (1995). "Bayesian Density Estimation and Inference Using Mixtures." Journal of the American Statistical Association, 90: 577–588.
  • –- (1998). "Computing Nonparametric Hierarchical Models." In Dey, D., Müller, P., and Sinha, D. (eds.), Practical nonparametric and semiparametric Bayesian statistics, 1–22. Springer - Verlag (Berlin, New York).
  • Fellingham, G. and Kottas, A. (2007). "Parametric and Nonparametric Bayesian Methods to Model Health Insurance Claims Costs." University of California at Santa Cruz, Department of Applied Math and Statistics Technical Reports.
  • Ferguson, T. (1973). "A Bayesian Analysis of Some Nonparametric Problems." Annals of Statistics, 1: 209–230.
  • –- (1974). "Prior Distributions on Spaces of Probability Measures." Annals of Statistics, 2: 615–629.
  • Fruehwirth-Schnatter, S. (1994). "Data augmentation and dynamic linear models." Journal of Time Series Analysis, 15: 183–202.
  • Gelfand, A., Kottas, A., and MacEachern, S. (2005). "Bayesian Nonparametric Spatial Modeling With Dirichlet Process Mixing." Journal of the American Statistical Association, 100: 1021–1035.
  • Green, P. and Richardson, S. (2001). "Modelling Heterogeneity With and Without the Dirichlet Process." Scandinavian Journal of Statistics, 28: 355–375.
  • Griffin, J. (2007). "The Ornstein-Uhlenbeck Dirichlet Process and other time-varying processes for Bayesian nonparametric inference." University of Kent, Technical Reports.
  • Griffin, J. and Steel, M. (2006). "Order-Based Dependent Dirichlet Processes." Journal of the American Statistical Association, 101: 179–194.
  • –- (2007). "Nonparametric Inference in Time Series Problems." mimeo, Dept. of Statistics, University of Warwick.
  • Hirano, K. (2002). "Semiparametric Bayesian Inference in Autoregressive Panel Data Models." Econometrica, 7: 781–799.
  • Huang, Z., Chen, H., Hsu, C.-J., and Chen, A. (2004). "Credit rating analysis with support vector machines and neural networks: a market comparative study." Decision Support Systems, 37: 543–558.
  • III, C. L., Carpenter, S., and Stow, C. (1998). "Forecasting PCB" Concentrations in Lake Michigan Salmonids: A Dynamic Linear Model Approach. Ecological Applications, 8: 659–668.
  • Ishwaran, H. and James, L. (2001). "Gibbs Sampling Methods for Stick-Breaking Priors." Journal of the American Statistical Association, 96: 161–173.
  • Kacperczyk, M., Damien, P., and Walker, S. (2004). "A new class of Bayesian Semiparametric Models with Applications to Option Pricing." Sauder School of Business, University of Michigan. Available at Social Science Research Network (SSRN).
  • Kalman, R. (1960). "A new approach to linear filtering and prediction problems." Transactions of the ASME - Journal of Basic Engineering, 82: 35–45.
  • Kim, C.-J. (1994). "Dynamic linear models with Markov-switching." Journal of Econometrics, 60: 1–22.
  • Kottas, A., Branco, M., and Gelfand, A. (2002). "A Nonparametric Bayesian Modeling Approach for Cytogenetic Dosimetry." Biometrics, 58: 593–600.
  • Laws, D. and O'Hagan, A. (2002). "A Hierarchical Bayes Model for Multilocation Auditing." 51: 431–450.
  • Lo, A. (1984). "On a class of Bayesian Nonparametric Estimates: I. Density Estimates." Annals of Statistics, 12: 351–357.
  • MacEachern, S. (1994). "Estimating Normal Means with a Conjugate Style Dirichlet Process Prior." Commnunications in Statistics, Part B - Simulation and Computation, 23(460): 727–741.
  • –- (1999). "Dependent Nonparametric Processes." In ASA Proceedings of the Section on Bayesian Statistical Science, 50–55.
  • MacEachern, S. and Müller, P. (1998). "Estimating Mixture of Dirichlet Process Models." Journal of Computational and Graphical Statistics, 7: 223–238.
  • Medvedovic, M. and Sivaganesan, S. (2002). "Bayesian Infinite Mixture Model-Based Clustering of Gene Expression Profiles." Bioinformatics, 18: 1194–1206.
  • Müller, P., Quintana, F., and Rosner, G. (2004). "Hierarchical Meta-Analysis over Related Non-parametric Bayesian Models." Journal of the Royal Statistical Society - Series B, 66: 735–749.
  • Neal, R. (2000). "Markov Chain Sampling Methods for Dirichlet Process Mixture Models." Journal of Computational and Graphical Statistics, 9: 249–265.
  • Pennell, M. and Dunson, D. (2008). "Nonparametric Bayes Testing of Changes in a Response Distribution with an Ordinal Predictor." Biometrics, to appear.
  • Pesaran, M. H., Smith, R. P., and Im, K.-S. (1995). "Dynamic Linear Models for Heterogeneous Panels." Faculty of Economics, University of Cambridge, (Cambridge Working Papers in Economics).
  • Reiss, R. and Thomas, M. (1999). "A new class of Bayesian estimator in Paretian excess-of-loss reinsurance." ASTIN Bulletin, 29: 339–349.
  • Richardson, S. and Green, P. (1997). "On Bayesian Analysis of Mixtures with an Unknown Number of Components." Journal of the Royal Statistical Society - Series B, 59(4): 731–792.
  • Rodriguez, A., Dunson, D. B., and Gelfand, A. E. (2008). "The nested Dirichlet process." Journal of the American Statistical Association, to appear with Discussion.
  • Sethuraman, J. (1994). "A Constructive Definition of Dirichlet Priors." Statistica Sinica, 4: 639–650.
  • Silverman, B. (1986). Density Estimation. Chapman and Hall, London.
  • Smyth, G. and Jorgensen, B. (2002). "Fitting Tweedie's compound Poisson model to insurance claims data: dispersion modelling." ASTIN Bulletin, 32(1): 143–157.
  • Tang, Y. and Ghosal, S. (2007). "A Consistent Nonparametric Bayesian Procedure for Estimating Autoregressive Conditional Densities." Computational Statistics & Data Analysis, 51: 4424–4437.
  • West, M. (1995). "Bayesian Time Series: Models and Computations for the Analysis of Time Series in the Physical Sciences." In XV Workshop on Maximum Entropy and Bayesian Methods. Oxford University Press.
  • –- (1997). "Time Series Decomposition." Biometrika, 84: 489–494.
  • West, M. and Harrison, J. (1997). Bayesian Forecasting and Dynamic Models. Reading, MA: Springer - Verlag, New York, second edition.
  • West, M., Prado, R., and Krystal, A. (1999). "Evaluation and comparison of EEG traces: Latent Structure in Non-Stationary Time Series." Journal of the American Statistical Association, 94: 1083–1095.