The Problem of Negative Estimates of Variance Components

Abstract

The usefulness of variance component techniques is frequently limited by the occurrence of negative estimates of essentially positive parameters. This paper uses a restricted maximum likelihood principle to remove this objectionable characteristic for certain experimental models. Section 2 discusses certain necessary results from the theory of non-linear programming. Section 3 derives specific formulae for estimating the variance components of the random one-way and two-way classification models. The problem of determining the precision of instruments in the two instrument case is dealt with in section 4, and a surprising though not unreasonable answer is obtained. The remaining sections provide an algorithm for solving the problem of negative estimates of variance components for all random effects models whose expected mean square column may be thought of as forming a mathematical tree in a certain sense. The algorithm is as follows: Consider the minimum mean square in the entire array; if this mean square is the root of the tree then equate it to its expectation. If the minimum mean square is not the root then pool it with its predecessor. In either case the problem is reduced to an identical one having one less variable, and hence in a finite number of steps the process will yield estimates of the variance components. These estimates are non-negative and have a maximum likelihood property.