- Bayesian Anal.
- Volume 11, Number 3 (2016), 885-911.
Efficient Metropolis–Hastings Proposal Mechanisms for Bayesian Regression Tree Models
Bayesian regression trees are flexible non-parametric models that are well suited to many modern statistical regression problems. Many such tree models have been proposed, from the simple single-tree model to more complex tree ensembles. Their nonparametric formulation allows one to model datasets exhibiting complex non-linear relationships between the model predictors and observations. However, the mixing behavior of the Markov Chain Monte Carlo (MCMC) sampler is sometimes poor, frequently suffering from local mode stickiness and poor mixing. This is because existing Metropolis–Hastings proposals do not allow for efficient traversal of the model space. We develop novel Metropolis–Hastings proposals that account for the topological structure of regression trees. The first is a novel tree rotation proposal that only requires local changes to the regression tree structure, yet efficiently traverses disparate regions of the model space along contours of high likelihood. The second is a rule perturbation proposal which can be seen as an efficient variation of the change proposal found in existing literature. We implement these samplers and demonstrate their effectiveness on a prediction problem from computer experiments, a test function where structural tree variability is needed to fully explore the posterior and data from a heart rate study.
Bayesian Anal., Volume 11, Number 3 (2016), 885-911.
First available in Project Euclid: 7 March 2016
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Pratola, Matthew T. Efficient Metropolis–Hastings Proposal Mechanisms for Bayesian Regression Tree Models. Bayesian Anal. 11 (2016), no. 3, 885--911. doi:10.1214/16-BA999. https://projecteuclid.org/euclid.ba/1457383101
- Related item: Robert B. Gramacy (2016). Comment on Article by Pratola. Bayesian Anal. Vol. 11, Iss. 3, 913–919.
- Related item: Christopher M. Hans (2016). Comment on Article by Pratola. Bayesian Anal. Vol. 11, Iss. 3, 921–927.
- Related item: Oksana A. Chkrebtii, Scotland Leman, Andrew Hoegh, Reihaneh Entezari, Radu V. Craiu, Jeffrey S. Rosenthal, Abdolreza Mohammadi, Maurits Kaptein, Luca Martino, Rafael B. Stern, Francisco Louzada (2016). Contributed Discussion on Article by Pratola. Bayesian Anal. Vol. 11, Iss. 3, 929–943.
- Related item: Matthew T. Pratola (2016). Rejoinder. Bayesian Anal. Vol. 11, Iss. 3, 945–955.
- Supplementary Material of “Efficient Metropolis–Hastings Proposal Mechanisms for Bayesian Regression Tree Models”.