Bayes Rules for a Common Multiple Comparisons Problem and Related Student- $t$ Problems

Abstract

The paper is mainly concerned with the following multiple comparisons problem in the analysis of variance setting. In a balanced experiment $n$ treatments are to be compared. Each of the $\frac{1}{2}n(n - 1)$ pairwise comparisons is to be made, adjudging each difference as "positive", "negative", or "not significant"; overall decisions involving intransitivities are barred. The loss for each difference is proportional to the error; if a difference is asserted incorrectly the loss has proportionality constant $c_1$, if "not-significant" is the incorrect conclusion the proportionality constant is $c_0$; where $c_1 = k_1 + k_0, c_0 = k_0$ and $k_1 > k_0 > 0$. Total loss for the experiment is taken as the sum of the $\frac{1}{2}n(n - 1)$ component losses. The Bayes rule for any prior distribution is shown as a result to consist in the simultaneous application of Bayes rules to the $\frac{1}{2}n(n - 1)$ component problems. Each of these in turn is shown similarly to consist in the simultaneous application of Bayes rules to two subcomponent problems. The subcomponent Bayes rule for a normal prior density of treatment means is explicitly derived. The dependencies of the solution on the variance of the prior density, the degrees of freedom and the loss ratio $k_1/k_0$ are discussed. A principal finding is that the Bayes solution for the multiple comparisons problem corresponds to a tolerated error probability "of the first kind" for each single difference, that is independent of the number of treatments being compared.