## Bernoulli

### Leading the field: Fortune favors the bold in Thurstonian choice models

#### Abstract

Schools with the highest average student performance are often the smallest schools; localities with the highest rates of some cancers are frequently small; and the effects observed in clinical trials are likely to be largest for the smallest numbers of subjects. Informal explanations of this “small-schools phenomenon” point to the fact that the sample means of smaller samples have higher variances. But this cannot be a complete explanation: If we draw two samples from a diffuse distribution that is symmetric about some point, then the chance that the smaller sample has larger mean is 50%. A particular consequence of results proved below is that if one draws three or more samples of different sizes from the same normal distribution, then the sample mean of the smallest sample is most likely to be highest, the sample mean of the second smallest sample is second most likely to be highest, and so on; this is true even though for any pair of samples, each one of the pair is equally likely to have the larger sample mean. The same effect explains why heteroscedasticity can result in misleadingly small nominal $p$-values in nonparametric tests of association.

Our conclusions are relevant to certain stochastic choice models, including the following generalization of Thurstone’s Law of Comparative Judgment. There are $n$ items. Item $i$ is preferred to item $j$ if $Z_{i}<Z_{j}$, where $Z$ is a random $n$-vector of preference scores. Suppose $\mathbb{P}\{Z_{i}=Z_{j}\}=0$ for $i\ne j$, so there are no ties. Item $k$ is the favorite if $Z_{k}<\min_{i\ne k}Z_{i}$. Let $p_{i}$ denote the chance that item $i$ is the favorite. We characterize a large class of distributions for $Z$ for which $p_{1}>p_{2}>\cdots>p_{n}$. Our results are most surprising when $\mathbb{P}\{Z_{i}<Z_{j}\}=\mathbb{P}\{Z_{i}>Z_{j}\}=\frac{1}{2}$ for $i\ne j$, so neither of any two items is likely to be preferred over the other in a pairwise comparison. Then, under suitable assumptions, $p_{1}>p_{2}>\cdots>p_{n}$ when the variability of $Z_{i}$ decreases with $i$ in an appropriate sense. Our conclusions echo the proverb “Fortune favors the bold.”

#### Article information

Source
Bernoulli, Volume 25, Number 1 (2019), 26-46.

Dates
Revised: December 2016
First available in Project Euclid: 12 December 2018

Permanent link to this document
https://projecteuclid.org/euclid.bj/1544605237

Digital Object Identifier
doi:10.3150/17-BEJ930

Mathematical Reviews number (MathSciNet)
MR3892310

Zentralblatt MATH identifier
07007198

#### Citation

Evans, Steven N.; Rivest, Ronald L.; Stark, Philip B. Leading the field: Fortune favors the bold in Thurstonian choice models. Bernoulli 25 (2019), no. 1, 26--46. doi:10.3150/17-BEJ930. https://projecteuclid.org/euclid.bj/1544605237

#### References

• [1] Agrawal, S. and Goyal, N. (2011). Analysis of Thompson sampling for the multi-armed bandit problem. CoRR. Available at arXiv/abs/1111.1797.
• [2] Böckenholt, U. (1993). Applications of Thurstonian models to ranking data. In Probability Models and Statistical Analyses for Ranking Data (Amherst, MA, 1990). Lect. Notes Stat. 80 157–172. New York: Springer.
• [3] Böckenholt, U. (2006). Thurstonian-based analyses: Past, present, and future utilities. Psychometrika 71 615–629.
• [4] Craig, P. (2008). A new reconstruction of multivariate normal orthant probabilities. J. R. Stat. Soc. Ser. B Stat. Methodol. 70 227–243.
• [5] Daniels, H.E. (1950). Rank correlation and population models. J. Roy. Statist. Soc. Ser. B. 12 171–181.
• [6] de Haan, L. and Ferreira, A. (2006). Extreme Value Theory. Springer Series in Operations Research and Financial Engineering: An Introduction. New York: Springer.
• [7] Diaconis, P. (1988). Group Representations in Probability and Statistics. Institute of Mathematical Statistics Lecture Notes – Monograph Series 11. Hayward, CA: IMS.
• [8] Fill, J.A. (1996). An exact formula for the move-to-front rule for self-organizing lists. J. Theoret. Probab. 9 113–160.
• [9] Fligner, M.A. and Verducci, J.S., eds. (1993) Probability Models and Statistical Analyses for Ranking Data. Lecture Notes in Statistics 80. New York: Springer. Papers from the conference held at the University of Massachusetts, Amherst, Massachusetts, June 8–13, 1990.
• [10] Gassmann, H.I., Deák, I. and Szántai, T. (2002). Computing multivariate normal probabilities: A new look. J. Comput. Graph. Statist. 11 920–949.
• [11] Gumbel, E.J. (2004). Statistics of Extremes. Mineola, NY: Dover.
• [12] Kotz, S. and Nadarajah, S. (2000). Extreme Value Distributions. Theory and Applications. London: Imperial College Press.
• [13] Laslier, J.-F. (2006). Strategic Approval Voting in a Large Electorate. Marseille, France: Institut D’Economie Publique. IDEP Working Papers No. 405. Available at http://halshs.archives-ouvertes.fr/docs/00/12/17/51/PDF/stratapproval4.pdf.
• [14] Luce, R.D. (1959). Individual Choice Behavior: A Theoretical Analysis. New York: Wiley; London: Chapman & Hall.
• [15] Marden, J.I. (1995). Analyzing and Modeling Rank Data. Monographs on Statistics and Applied Probability 64. London: Chapman & Hall.
• [16] Miwa, T., Hayter, A.J. and Kuriki, S. (2003). The evaluation of general non-centred orthant probabilities. J. R. Stat. Soc. Ser. B Stat. Methodol. 65 223–234.
• [17] Plackett, R.L. (1975). The analysis of permutations. J. Roy. Statist. Soc. Ser. C Appl. Statist. 24 193–202.
• [18] Resnick, S.I. (2008). Extreme Values, Regular Variation and Point Processes. Springer Series in Operations Research and Financial Engineering. New York: Springer. Reprint of the 1987 original.
• [19] Rivest, R. (1976). On self-organizing sequential search heuristics. Comm. ACM 19 63–67.
• [20] Savage, I.R. (1957). Contributions to the theory of rank order statistics – the “trend” case. Ann. Math. Statist 28 968–977.
• [21] Silverberg, A.R. (1980). Statistical Models for $Q$-Permutations. Ann Arbor, MI: ProQuest LLC. Thesis (Ph.D.) – Princeton University.
• [22] Silverberg, A.R. (1984). Statistical models for $q$-permutations. Proc. Biopharm. Sec. Amer. Statist. Assoc. 107–112.
• [23] Thompson, W.R. (1933). On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25 285–294.
• [24] Thurstone, L.L. (1927). A law of comparative judgement. Psychological Reviews 34 273–286.
• [25] Thurstone, L.L. (1931). Rank order as a psycho-physical method. J. Exp. Psychol. 14 187–201.
• [26] Wainer, H. (2007). The most dangerous equation. Am. Sci. 95 249–256. Available at http://www.americanscientist.org/issues/feature/2007/3/the-most-dangerous-equation/1.
• [27] Walther, G. (1997). Absence of correlation between the solar neutrino flux and the sunspot number. Phys. Rev. Lett. 79 4522–4524.
• [28] Walther, G. (1999). On the solar-cycle modulation of the Homestake solar neutrino capture rate and the shuffle test. Ap. J. 513 990–996.
• [29] Yellott, J.I. Jr. (1977). The relationship between Luce’s choice axiom, Thurstone’s theory of comparative judgment, and the double exponential distribution. J. Math. Psych. 15 109–144.