Open Access
August 2004 Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
Iain M. Johnstone, Bernard W. Silverman
Ann. Statist. 32(4): 1594-1649 (August 2004). DOI: 10.1214/009053604000000030


An empirical Bayes approach to the estimation of possibly sparse sequences observed in Gaussian white noise is set out and investigated. The prior considered is a mixture of an atom of probability at zero and a heavy-tailed density γ, with the mixing weight chosen by marginal maximum likelihood, in the hope of adapting between sparse and dense sequences. If estimation is then carried out using the posterior median, this is a random thresholding procedure. Other thresholding rules employing the same threshold can also be used. Probability bounds on the threshold chosen by the marginal maximum likelihood approach lead to overall risk bounds over classes of signal sequences of length n, allowing for sparsity of various kinds and degrees. The signal classes considered are “nearly black” sequences where only a proportion η is allowed to be nonzero, and sequences with normalized ℓp norm bounded by η, for η>0 and 0<p≤2. Estimation error is measured by mean qth power loss, for 0<q≤2. For all the classes considered, and for all q in (0,2], the method achieves the optimal estimation rate as n→∞ and η→0 at various rates, and in this sense adapts automatically to the sparseness or otherwise of the underlying signal. In addition the risk is uniformly bounded over all signals. If the posterior mean is used as the estimator, the results still hold for q>1. Simulations show excellent performance. For appropriately chosen functions γ, the method is computationally tractable and software is available. The extension to a modified thresholding method relevant to the estimation of very sparse sequences is also considered.


Download Citation

Iain M. Johnstone. Bernard W. Silverman. "Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences." Ann. Statist. 32 (4) 1594 - 1649, August 2004.


Published: August 2004
First available in Project Euclid: 4 August 2004

zbMATH: 1047.62008
MathSciNet: MR2089135
Digital Object Identifier: 10.1214/009053604000000030

Primary: 62C12
Secondary: 62G05 , 62G08

Keywords: Adaptivity , Empirical Bayes , sequence estimation , Sparsity , thresholding

Rights: Copyright © 2004 Institute of Mathematical Statistics

Vol.32 • No. 4 • August 2004
Back to Top