In modern high-throughput data analysis, researchers perform a large number of statistical tests, expecting to find perhaps a small fraction of significant effects against a predominantly null background. Higher Criticism (HC) was introduced to determine whether there are any nonzero effects; more recently, it was applied to feature selection, where it provides a method for selecting useful predictive features from a large body of potentially useful features, among which only a rare few will prove truly useful.

In this article, we review the basics of HC in both the testing and feature selection settings. HC is a flexible idea, which adapts easily to new situations; we point out simple adaptions to clique detection and bivariate outlier detection. HC, although still early in its development, is seeing increasing interest from practitioners; we illustrate this with worked examples. HC is computationally effective, which gives it a nice leverage in the increasingly more relevant “Big Data” settings we see today.

We also review the underlying theoretical “ideology” behind HC. The Rare/Weak (RW) model is a theoretical framework simultaneously controlling the size and prevalence of useful/significant items among the useless/null bulk. The RW model shows that HC has important advantages over better known procedures such as False Discovery Rate (FDR) control and Family-wise Error control (FwER), in particular, certain optimality properties. We discuss the rare/weak phase diagram, a way to visualize clearly the class of RW settings where the true signals are so rare or so weak that detection and feature selection are simply impossible, and a way to understand the known optimality properties of HC.

Part V of the second edition of Pierre Rémond de Montmort’s Essay d’analyse sur les jeux de hazard published in 1713 contains correspondence on probability problems between Montmort and Nicolaus Bernoulli. This correspondence begins in 1710. The last published letter, dated November 15, 1713, is from Montmort to Nicolaus Bernoulli. There is some discussion of the strategy of play in the card game Le Her and a bit of news that Montmort’s friend Waldegrave in Paris was going to take care of the printing of the book. From earlier correspondence between Bernoulli and Montmort, it is apparent that Waldegrave had also analyzed Le Her and had come up with a mixed strategy as a solution. He had also suggested working on the “problem of the pool,” or what is often called Waldegrave’s problem. The Universitätsbibliothek Basel contains an additional forty-two letters between Bernoulli and Montmort written after 1713, as well as two letters between Bernoulli and Waldegrave. The letters are all in French, and here we provide translations of key passages. The trio continued to discuss probability problems, particularly Le Her which was still under discussion when the Essay d’analyse went to print. We describe the probability content of this body of correspondence and put it in its historical context. We also provide a proper identification of Waldegrave based on manuscripts in the Archives nationales de France in Paris.

This article provides a completion to theories of information based on entropy, resolving a longstanding question in its axiomatization as proposed by Shannon and pursued by Jaynes. We show that Shannon’s entropy function has a complementary dual function which we call “extropy.” The entropy and the extropy of a binary distribution are identical. However, the measure bifurcates into a pair of distinct measures for any quantity that is not merely an event indicator. As with entropy, the maximum extropy distribution is also the uniform distribution, and both measures are invariant with respect to permutations of their mass functions. However, they behave quite differently in their assessments of the refinement of a distribution, the axiom which concerned Shannon and Jaynes. Their duality is specified via the relationship among the entropies and extropies of course and fine partitions. We also analyze the extropy function for densities, showing that relative extropy constitutes a dual to the Kullback–Leibler divergence, widely recognized as the continuous entropy measure. These results are unified within the general structure of Bregman divergences. In this context they identify half the $L_{2}$ metric as the extropic dual to the entropic directed distance. We describe a statistical application to the scoring of sequential forecast distributions which provoked the discovery.

As increasingly complex hypothesis-testing scenarios are considered in many scientific fields, analytic derivation of null distributions is often out of reach. To the rescue comes Monte Carlo testing, which may appear deceptively simple: as long as you can sample test statistics under the null hypothesis, the $p$-value is just the proportion of sampled test statistics that exceed the observed test statistic. Sampling test statistics is often simple once you have a Monte Carlo null model for your data, and defining some form of randomization procedure is also, in many cases, relatively straightforward. However, there may be several possible choices of a randomization null model for the data and no clear-cut criteria for choosing among them. Obviously, different null models may lead to very different $p$-values, and a very low $p$-value may thus occur due to the inadequacy of the chosen null model. It is preferable to use assumptions about the underlying random data generation process to guide selection of a null model. In many cases, we may order the null models by increasing preservation of the data characteristics, and we argue in this paper that this ordering in most cases gives increasing $p$-values, that is, lower significance. We denote this as the null complexity principle. The principle gives a better understanding of the different null models and may guide in the choice between the different models.

Indirect inference (II) is a methodology for estimating the parameters of an intractable (generative) model on the basis of an alternative parametric (auxiliary) model that is both analytically and computationally easier to deal with. Such an approach has been well explored in the classical literature but has received substantially less attention in the Bayesian paradigm. The purpose of this paper is to compare and contrast a collection of what we call parametric Bayesian indirect inference (pBII) methods. One class of pBII methods uses approximate Bayesian computation (referred to here as ABC II) where the summary statistic is formed on the basis of the auxiliary model, using ideas from II. Another approach proposed in the literature, referred to here as parametric Bayesian indirect likelihood (pBIL), uses the auxiliary likelihood as a replacement to the intractable likelihood. We show that pBIL is a fundamentally different approach to ABC II. We devise new theoretical results for pBIL to give extra insights into its behaviour and also its differences with ABC II. Furthermore, we examine in more detail the assumptions required to use each pBII method. The results, insights and comparisons developed in this paper are illustrated on simple examples and two other substantive applications. The first of the substantive examples involves performing inference for complex quantile distributions based on simulated data while the second is for estimating the parameters of a trivariate stochastic process describing the evolution of macroparasites within a host based on real data. We create a novel framework called Bayesian indirect likelihood (BIL) that encompasses pBII as well as general ABC methods so that the connections between the methods can be established.

Instrumental variables analysis using genetic markers as instruments is now a widely used technique in epidemiology and biostatistics. As single markers tend to explain only a small proportion of phenotypic variation, there is increasing interest in using multiple genetic markers to obtain more precise estimates of causal parameters. Structural mean models (SMMs) are semiparametric models that use instrumental variables to identify causal parameters. Recently, interest has started to focus on using these models with multiple instruments, particularly for multiplicative and logistic SMMs. In this paper we show how additive, multiplicative and logistic SMMs with multiple orthogonal binary instrumental variables can be estimated efficiently in models with no further (continuous) covariates, using the generalised method of moments (GMM) estimator. We discuss how the Hansen J-test can be used to test for model misspecification, and how standard GMM software routines can be used to fit SMMs. We further show that multiplicative SMMs, like the additive SMM, identify a weighted average of local causal effects if selection is monotonic. We use these methods to reanalyse a study of the relationship between adiposity and hypertension using SMMs with two genetic markers as instruments for adiposity. We find strong effects of adiposity on hypertension.

Richard Olshen was born in Portland, Oregon, on May 17, 1942. Richard spent his early years in Chevy Chase, Maryland, but has lived most of his life in California. He received an A.B. in Statistics at the University of California, Berkeley, in 1963, and a Ph.D. in Statistics from Yale University in 1966, writing his dissertation under the direction of Jimmie Savage and Frank Anscombe. He served as Research Staff Statistician and Lecturer at Yale in 1966–1967.

Richard accepted a faculty appointment at Stanford University in 1967, and has held tenured faculty positions at the University of Michigan (1972–1975), the University of California, San Diego (1975–1989), and Stanford University (since 1989). At Stanford, he is Professor of Health Research and Policy (Biostatistics), Chief of the Division of Biostatistics (since 1998) and Professor (by courtesy) of Electrical Engineering and of Statistics. At various times, he has had visiting faculty positions at Columbia, Harvard, MIT, Stanford and the Hebrew University.

Richard’s research interests are in statistics and mathematics and their applications to medicine and biology. Much of his work has concerned binary tree-structured algorithms for classification, regression, survival analysis and clustering. Those for classification and survival analysis have been used with success in computer-aided diagnosis and prognosis, especially in cardiology, oncology and toxicology. He coauthored the 1984 book Classification and Regression Trees (with Leo Brieman, Jerome Friedman and Charles Stone) which gives motivation, algorithms, various examples and mathematical theory for what have come to be known as CART algorithms. The approaches to tree-structured clustering have been applied to problems in digital radiography (with Stanford EE Professor Robert Gray) and to HIV genetics, the latter work including studies on single nucleotide polymorphisms, which has helped to shed light on the presence of hypertension in certain subpopulations of women.

Richard also has a long-standing interest in the analyses of longitudinal data. This includes a detailed study of the pharmacokinetics of intracavitary chemotherapy with systemic rescue (with Stephen Howell and John Rice). Related efforts have focused on “mature walking,” concomitants of high cholesterol, and aspects of glomerular filtration in patients with nephrotic disorders (with Bryan Myers). With the late David Sutherland, Edmund Biden and Marilynn Wyatt, he coauthored the monograph The Development of Mature Walking. Richard’s other stochastic-statistical interests include exchangeability, conditional significance levels of particular test statistics, CART-like estimators in regression and successive standardization of rectangular arrays of numbers.

Richard is an elected Fellow of the IMS, the AAAS, the ASA and the IEEE. He is a former Guggenheim Fellow and has been a Research Scholar in Cancer of the American Cancer Society.

Nancy Flournoy was born in Long Beach, California, on May 4, 1947. After graduating from Polytechnic School in Pasadena in 1965, she earned a B.S. (1969) and M.S. (1971) in biostatistics from UCLA. Between her bachelors and masters degrees, she worked as a Statistician I for Regional Medical Programs at UCLA. After receiving her master’s degree, she spend three years at the Southwest Laboratory for Education Research and Development in Seal Beach, California. Flournoy joined the Seattle team pioneering bone marrow transplantation in 1973. She moved with the transplant team into the newly formed Fred Hutchinson Cancer Research Center in 1975 as Director of Clinical Statistics, where she supervised a group responsible for the design and analysis of about 80 simultaneous clinical trials. To support the Clinical Division, she supervised the development of an interdisciplinary shared data software system. She recruited Leonard B. Hearne to create this database management system in 1975 (and married him in 1978). While at the Cancer Center, she was also at the University of Washington, where she received her doctorate in biomathematics in 1982. She became the first female director of the program in statistics at the National Science Foundation (NSF) in 1986. She received service awards from the NSF in 1988 and the National Institute of Statistical Science in 2006 for facilitating interdisciplinary research. Flournoy joined the Department of Mathematics and Statistics at American University in 1988. She moved as department chair to the University of Missouri in 2002, where she became Curators’ Distinguished Professor in 2012.

While at the Cancer Center, Flournoy documented the graft-versus-leukemia effect in humans and discovered a source of frequent lethal viral infections in the bone marrow transplant patients. Later she was influential in developing adaptive experimental designs. Her numerous honors include fellow of the Institute of Mathematical Statistics (1990), the American Statistical Association (1992), the World Academy of Arts and Sciences (1992) and the American Academy for the Advancement of Science (1993). She has received the COPSS Scott (2000) and David (2007) awards, and the Norwood (2012) award from the University of Alabama.