The Annals of Applied Probability

Optimal scaling of MaLa for nonlinear regression

Laird Arnault Breyer, Mauro Piccioni, and Sergio Scarlatti

Full-text: Open access

Enhanced PDF (209 KB)

Abstract

We address the problem of simulating efficiently from the posterior distribution over the parameters of a particular class of nonlinear regression models using a Langevin–Metropolis sampler. It is shown that as the number N of parameters increases, the proposal variance must scale as N−1/3 in order to converge to a diffusion. This generalizes previous results of Roberts and Rosenthal [J. R. Stat. Soc. Ser. B Stat. Methodol. 60 (1998) 255–268] for the i.i.d. case, showing the robustness of their analysis.

Article information

Source
Ann. Appl. Probab., Volume 14, Number 3 (2004), 1479-1505.

Dates
First available in Project Euclid: 13 July 2004

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1089736293

Digital Object Identifier
doi:10.1214/105051604000000369

Mathematical Reviews number (MathSciNet)
MR2071431

Zentralblatt MATH identifier
1048.62062

Subjects
Primary: 60F17: Functional limit theorems; invariance principles
Secondary: 60F05: Central limit and other weak theorems 60F10: Large deviations

Keywords
Bayesian nonlinear regression Markov chain Monte Carlo Hastings–Metropolis Langevin diffusion propagation of chaos

Citation

Breyer, Laird Arnault; Piccioni, Mauro; Scarlatti, Sergio. Optimal scaling of MaLa for nonlinear regression. Ann. Appl. Probab. 14 (2004), no. 3, 1479--1505. doi:10.1214/105051604000000369. https://projecteuclid.org/euclid.aoap/1089736293


Export citation

References

  • Ben Arous, G. and Brunaud, M. (1990). Methode de Laplace: Etude variationelle des fluctuations de diffusions de type “champs moyen.” Stochastics 31 79–144.\goodbreak
    Zentralblatt MATH: 0705.60046
    Digital Object Identifier: doi:10.1080/03610919008833649
  • Breyer, L. A. and Roberts, G. O. (2000). From Metropolis to diffusions: Gibbs states and optimal scaling. Stochastic Process. Appl. 90 181–206.
    Zentralblatt MATH: 1047.60065
    Digital Object Identifier: doi:10.1016/S0304-4149(00)00041-7
  • Christensen, O. F., Roberts, G. O. and Rosenthal, J. S. (2003). Scaling limits for the transient phase of local Metropolis–Hastings algorithms. Available at probability.ca/jeff/research.html.
  • Csiszar, I. (1975). I-divergence geometry of probability distributions and minimization problems. Ann. Probab. 3 146–158.
    Zentralblatt MATH: 0318.60013
    Digital Object Identifier: doi:10.1214/aop/1176996454
    Project Euclid: euclid.aop/1176996454
  • Csiszar, I. (1984). Sanov property, generalized I-projection and a conditional limit theorem. Ann. Probab. 12 768–793.
    Zentralblatt MATH: 0544.60011
    Digital Object Identifier: doi:10.1214/aop/1176993227
    Project Euclid: euclid.aop/1176993227
  • Dembo, A. and Zeitouni, O. (1998). Large Deviations Techniques and Applications. Springer, New York.
    Zentralblatt MATH: 0896.60013
  • Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes. Characterization and Convergence. Wiley, New York.
    Zentralblatt MATH: 0592.60049
  • Gamboa, F. (1999). New Bayesian methods for ill-posed problems. Statist. Decisions 17 315–337.
    Zentralblatt MATH: 0952.62027
  • Kusuoka, S. and Tamura, Y. (1984). Gibbs measures for mean field potentials. J. Fac. Sci. Tokyo Sect. IA Math. 31 223–245.
    Zentralblatt MATH: 0549.60099
  • Neal, R. (1996). Bayesian Learning for Neural Networks. Springer, New York.
    Zentralblatt MATH: 0888.62021
  • Petrov, V. V. (1995). Limit Theorems of Probability Theory. Oxford Univ. Press.
    Zentralblatt MATH: 0826.60001
  • Piccioni, M. and Scarlatti, S. (2000). Mean field models and propagation of chaos in feedforward neural networks. In Stochastic Processes, Physics and Geometry. New Interplays I (F. Gesztesy, ed.). Amer. Math. Soc., Providence, RI.
    Zentralblatt MATH: 1106.82320
  • Roberts, G. O., Gelman, A. and Gilks, W. R. (1997). Weak convergence and optimal scaling of random walk Metropolis algorithms. Ann. Appl. Probab. 7 110–120.
    Zentralblatt MATH: 0876.60015
    Digital Object Identifier: doi:10.1214/aoap/1034625254
    Project Euclid: euclid.aoap/1034625254
  • Roberts, G. O. and Rosenthal, J. (1998). Optimal scaling of discrete approximations to Langevin diffusions. J. R. Stat. Soc. Ser. B Stat. Methodol. 60 255–268.
    Zentralblatt MATH: 0913.60060
    Digital Object Identifier: doi:10.1111/1467-9868.00123
  • Roberts, G. O. and Rosenthal, J. (2001). Optimal scaling for various Metropolis–Hastings algorithms. Statist. Sci. 16 351–367.
    Zentralblatt MATH: 1127.65305
    Digital Object Identifier: doi:10.1214/ss/1015346320
    Project Euclid: euclid.ss/1015346320
  • Roberts, G. O. and Tweedie, R. L. (1996). Exponential convergence of Langevin diffusions and their discrete approximations. Bernoulli 2 341–363.
    Zentralblatt MATH: 0870.60027
    Digital Object Identifier: doi:10.2307/3318418
    Project Euclid: euclid.bj/1178291835
  • Rogers, L. C. G. and Williams, D. (1987). Diffusions, Markov Processes, and Martingales II. Wiley, Chichester.
    Zentralblatt MATH: 0627.60001
  • Sznitman, A. (1989). Topics in propagation of chaos. Ecole d'Eté de Probabilites de Saint-Flour XIX. Lecture Notes in Math. 1464 165–251. Springer, Berlin.

The Institute of Mathematical Statistics

The Annals of Applied Probability

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more