A comprehensive review of bias reduction methods for logistic regression

Abstract

The maximum likelihood estimator (MLE) for the unknown parameter vector in logistic regression is well known to be biased. There are many different approaches to reduce this bias including bias correction, adjustment of the score function or of the data itself, jackknifing, penalizing the likelihood, exact logistic regression, and the discriminant function approach. These approaches, as well as many different simulation studies comparing them, are reviewed here. Since the studies use very different parameter settings and sometimes contradict each other, no general recommendations can be given. However, most studies find that the bias of the MLE is substantial for small to medium samples, that the bias-corrected estimators tend to overcorrect in very small samples, and that Firth’s estimator, when considered, is the best choice.