On the Balanced Incomplete Block Design for Rankings

Abstract

A total of $nb$ judges rank $t$ objects $k$ at a time according to $n$ replications of a BIBD with $b$ blocks. The Durbin statistic is commonly used in this context and is equivalent to the usual analysis of variance on the rankings. The approach considered here is to introduce the notion of compatibility so as to define distances between incomplete rankings based on metrics on the space of complete rankings. Through this device we define a class of test statistics which includes the Durbin statistic as a special case, and derive their asymptotic distributions. This analysis also yields a new interpretation of the Durbin statistic.