Bayesian Analysis

Estimating a bivariate linear relationship

David Leonard

Full-text: Open access


Solutions of the bivariate, linear errors-in-variables estimation problem with unspecified errors are expected to be invariant under interchange and scaling of the coordinates. The appealing model of normally distributed true values and errors is unidentified without additional information. I propose a prior density that incorporates the fact that the slope and variance parameters together determine the covariance matrix of the unobserved true values but is otherwise diffuse. The marginal posterior density of the slope is invariant to interchange and scaling of the coordinates and depends on the data only through the sample correlation coefficient and ratio of standard deviations. It covers the interval between the two ordinary least squares estimates but diminishes rapidly outside of it. I introduce the R package leivfor computing the posterior density, and I apply it to examples in astronomy and method comparison.

Article information

Bayesian Anal., Volume 6, Number 4 (2011), 727-754.

First available in Project Euclid: 13 June 2012

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Errors-in-variables Identifiability Measurement error Straight line fitting


Leonard, David. Estimating a bivariate linear relationship. Bayesian Anal. 6 (2011), no. 4, 727--754. doi:10.1214/11-BA627.

Export citation


  • Abramowitz, M. and Stegun, I. A. (eds.) (1964). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Number 55 in Applied Mathematics Series. Washington, D. C.: U. S. Government Printing Office. Tenth printing, 1972, with corrections.
  • Adcock, R. J. (1877). "Note on the Method of Least Squares." The Analyst, 4: 183–184.
  • –- (1878). "A Problem in Least Squares." The Analyst, 5: 53–54.
  • Altman, D. G. and Bland, J. M. (1983). "Measurement in Medicine: the Analysis of Method Comparison Studies." The Statistician, 32: 307–317.
  • Anderson, T. W. (1984). "Estimating Linear Statistical Relationships." The Annals of Statistics, 12: 1–45.
  • –- (1984). An Introduction to Multivariate Statistical Analysis. New York, NY: John Wiley & Sons, Inc., second edition.
  • Babu, G. J. and Feigelson, E. D. (1992). "Analytical and Monte Carlo Comparisons of Six Different Linear Least Squares Fits." Communications in Statistics–-Simulation and Computation, 21: 533–549.
  • Bland, J. M. and Altman, D. G. (1986). "Statistical Methods for Assessing Agreement Between Two Methods of Clinical Measurement." The Lancet, i: 307–310.
  • –- (1999). "Measuring Agreement in Method Comparison Studies." Statistical Methods in Medical Research, 8: 135–160.
  • Davison, A. C. and Hinkley, D. V. (1997). Bootstrap Methods and their Application. Cambridge, U. K.: Cambridge University Press.
  • Dunn, G. (2007). "Regression Models for Method Comparison Data." Journal of Biopharmaceutical Statistics, 17: 739–756.
  • Fisher, R. A. (1915). "Frequency Distribution of the Values of the Correlation Coefficient in Samples from an Indefinitely Large Population." Biometrika, 10: 507–521.
  • Frisch, R. (1934). "Statistical Confluence Analysis by Means of Complete Regression Systems." Technical Report 5, University Economics Institute, Oslo, Norway.
  • Fuller, W. A. (1987). Measurement Error Models. New York, NY: John Wiley & Sons, Inc.
  • Isobe, T., Feigelson, E. D., Akritas, M. G., and Babu, G. J. (1990). "Linear Regression in Astronomy. I." Astrophysical Journal, 364: 104–113.
  • Jaynes, E. T. (2003). Probability Theory: The Logic of Science. Cambridge, U. K.: Cambridge University Press. Edited by G. Larry Bretthorst.
  • Klepper, S. and Leamer, E. E. (1984). "Consistent Sets of Estimates for Regressions with Errors in All Variables." Econometrica, 52: 163–183.
  • Leonard, D. (2011). leiv: Bivariate Linear Errors-In-Variables Estimation. R package version 2.0-1.
  • Lindley, D. V. and El-Sayyad, G. M. (1968). "The Bayesian Estimation of a Linear Functional Relationship." Journal of the Royal Statistical Society. Series B (Methodological), 30: 190–202.
  • Madansky, A. (1959). "The Fitting of Straight Lines When Both Variables are Subject to Error." Journal of the American Statistical Association, 54: 173–205.
  • Olver, F. W. J., Lozier, D. W., Boisvert, R. F., and Clark, C. W. (eds.) (2010). NIST Handbook of Mathematical Functions. Cambridge, U. K.: Cambridge University Press. Online version available as Digital Library of Mathematical Functions from National Institute of Standards and Technology, release date 2011-08-29.
  • Pearson, K. (1901). "On Lines and Planes of Closest Fit to Systems of Points in Space." The London, Edinburgh and Dublin Philosophical Magazine and Journal of Science, Series 6, 2: 559–572.
  • Poirier, D. J. (1998). "Revising Beliefs in Nonidentified Models." Econometric Theory, 14: 483–509.
  • Polasek, W. and Krause, A. (1993). "Bayesian Regression Model with Simple Errors in Variables Structure." Journal of the Royal Statistical Society. Series D (The Statistician), 42: 571–580.
  • R Development Core Team (2011). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0.
  • Reiersøl, O. (1950). "Identifiability of a Linear Relation Between Variables which are Subject to Error." Econometrica, 18: 375–389.
  • Samuelson, P. A. (1942). "A Note on Alternative Regressions." Econometrica, 10: 80–83.
  • Schechter, P. L. (1980). "Mass-To-Light Ratios for Elliptical Galaxies." The Astronomical Journal, 85: 801–811.
  • Sprent, P. (1990). "Some History of Functional and Structural Relationships." In Brown, P. J. and Fuller, W. A. (eds.), Statistical Analysis of Measurement Error Models and Applications, Proceedings of the AMS-IMS-SIAM Joint Summer Research Conference, volume 112 of Contemporary Mathematics, 3–15. Providence, RI: American Mathematical Society.
  • Stefanski, L. A. (2000). "Measurement Error Models." Journal of the American Statistical Association, 95: 1353–1358.
  • Wald, A. (1940). "The Fitting of Straight Lines if Both Variables are Subject to Error." The Annals of Mathematical Statistics, 11: 284–300.
  • Warton, D. I., Wright, I. J., Falster, D. S., and Westoby, M. (2006). "Bivariate Line-Fitting Methods for Allometry." Biological Reviews, 81: 259–291.
  • Zellner, A. (1971). An Introduction to Bayesian Inference in Econometrics. New York, NY: John Wiley & Sons, Inc. Reprinted, 1996.