• Bernoulli
  • Volume 25, Number 2 (2019), 977-1012.

Are there needles in a moving haystack? Adaptive sensing for detection of dynamically evolving signals

Rui M. Castro and Ervin Tánczos

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


In this paper, we investigate the problem of detecting dynamically evolving signals. We model the signal as an $n$ dimensional vector that is either zero or has $s$ non-zero components. At each time step $t\in\mathbb{N}$ the nonzero components change their location independently with probability $p$. The statistical problem is to decide whether the signal is a zero vector or in fact it has non-zero components. This decision is based on $m$ noisy observations of individual signal components collected at times $t=1,\ldots,m$. We consider two different sensing paradigms, namely adaptive and non-adaptive sensing. For non-adaptive sensing, the choice of components to measure has to be decided before the data collection process started, while for adaptive sensing one can adjust the sensing process based on observations collected earlier. We characterize the difficulty of this detection problem in both sensing paradigms in terms of the aforementioned parameters, with special interest to the speed of change of the active components. In addition, we provide an adaptive sensing algorithm for this problem and contrast its performance to that of non-adaptive detection algorithms.

Article information

Bernoulli, Volume 25, Number 2 (2019), 977-1012.

Received: February 2017
Revised: November 2017
First available in Project Euclid: 6 March 2019

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

adaptive sensing dynamically evolving signals sequential experimental design sparse signals


Castro, Rui M.; Tánczos, Ervin. Are there needles in a moving haystack? Adaptive sensing for detection of dynamically evolving signals. Bernoulli 25 (2019), no. 2, 977--1012. doi:10.3150/17-BEJ1010.

Export citation


  • [1] Addario-Berry, L., Broutin, N., Devroye, L. and Lugosi, G. (2010). On combinatorial testing problems. Ann. Statist. 38 3063–3092.
  • [2] Baraud, Y. (2002). Non-asymptotic minimax rates of testing in signal detection. Bernoulli 8 577–606.
  • [3] Bayraktar, E. and Lai, L. (2015). Byzantine fault tolerant distributed quickest change detection. SIAM J. Control Optim. 53 575–591.
  • [4] Caromi, R., Xin, Y. and Lai, L. (2013). Fast multiband spectrum scanning for cognitive radio systems. IEEE Trans. Commun. 61 63–75.
  • [5] Castro, R.M. (2014). Adaptive sensing performance lower bounds for sparse signal detection and support estimation. Bernoulli 20 2217–2246.
  • [6] Castro, R.M. and Tánczos, E. (2015). Adaptive sensing for estimation of structured sparse signals. IEEE Trans. Inform. Theory 61 2060–2080.
  • [7] Castro, R.M. and Tánczos, E. (2017). Adaptive compressed sensing for support recovery of structured sparse sets. IEEE Trans. Inform. Theory 63 1535–1554.
  • [8] Donoho, D. and Jin, J. (2004). Higher criticism for detecting sparse heterogeneous mixtures. Ann. Statist. 32 962–994.
  • [9] Dragalin, V. (1996). A simple and effective scanning rule for a multi-channel system. Metrika 43 165–182.
  • [10] Enikeeva, F., Munk, A. and Werner, F. (2018). Bump detection in heterogeneous Gaussian regression. Bernoulli 24 1266–1306.
  • [11] Flenner, A. and Hewer, G. (2011). A Helmholtz principle approach to parameter free change detection and coherent motion using exchangeable random variables. SIAM J. Imaging Sci. 4 243–276.
  • [12] Gwadera, R., Atallah, M.J. and Szpankowski, W. (2005). Reliable detection of episodes in event sequences. Knowl. Inf. Syst. 7 415–437.
  • [13] Hadjiliadis, O., Zhang, H. and Poor, H.V. (2008). One shot schemes for decentralized quickest change detection. In 11th International Conference on Information Fusion 1–8.
  • [14] Haupt, J., Castro, R.M. and Nowak, R. (2011). Distilled sensing: Adaptive sampling for sparse detection and estimation. IEEE Trans. Inform. Theory 57 6222–6235.
  • [15] Huang, L., Kulldorff, M. and Gregorio, D. (2007). A spatial scan statistic for survival data. Biometrics 63 109–118, 311–312.
  • [16] Ingster, Y.I. (1997). Some problems of hypothesis testing leading to infinitely divisible distributions. Math. Methods Statist. 6 47–69.
  • [17] Ingster, Y.I. and Suslina, I.A. (2000). Minimax nonparametric hypothesis testing for ellipsoids and Besov bodies. ESAIM Probab. Stat. 4 53–135.
  • [18] Ingster, Y.I. and Suslina, I.A. (2002). On the detection of a signal with a known shape in a multichannel system. Zap. Nauchn. Sem. S.-Peterburg. Otdel. Mat. Inst. Steklov. (POMI) 294 88–112, 261.
  • [19] Joag-Dev, K. and Proschan, F. (1983). Negative association of random variables, with applications. Ann. Statist. 11 286–295.
  • [20] Klimko, E.M. and Yackel, J. (1975). Optimal search strategies for Wiener processes. Stochastic Process. Appl. 3 19–33.
  • [21] Kulldorff, M., Heffernan, R., Hartman, J., Assunçao, R. and Mostashari, F. (2005). A space–time permutation scan statistic for disease outbreak detection. PLoS Med. 2 216–224.
  • [22] Kulldorff, M., Huang, L. and Konty, K. (2009). A scan statistic for continuous data based on the normal probability model. Int. J. Health Geogr. 8 58.
  • [23] Li, H. (2009). Restless watchdog: Selective quickest spectrum sensing in multichannel cognitive radio systems. EURASIP J. Adv. Signal Process. 2009 Article ID: 417457.
  • [24] Luo, W. and Tay, W.P. (2013). Finding an infection source under the SIS model. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2930–2934.
  • [25] Malloy, M. and Nowak, R. (2011). On the limits of sequential testing in high dimensions. In Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR), 2011 1245–1249.
  • [26] Malloy, M.L. and Nowak, R.D. (2014). Sequential testing for sparse recovery. IEEE Trans. Inform. Theory 60 7862–7873.
  • [27] Neill, D.B. and Moore, A.W. (2004). A fast multi-resolution method for detection of significant spatial disease clusters. In Advances in Neural Information Processing Systems 16 651–658. MIT Press.
  • [28] Pawitan, Y., Michiels, S., Koscielny, S., Gusnanto, A. and Ploner, A. (2005). False discovery rate, sensitivity and sample size for microarray studies. Bioinformatics 21 3017–3024.
  • [29] Phoha, V.V. (2007). Internet Security Dictionary. Springer Science & Business Media.
  • [30] Raghavan, V. and Veeravalli, V.V. (2010). Quickest change detection of a Markov process across a sensor array. IEEE Trans. Inform. Theory 56 1961–1981.
  • [31] Shah, D. and Zaman, T. (2011). Rumors in a network: Who’s the culprit? IEEE Trans. Inform. Theory 57 5163–5181.
  • [32] Thompson, D.R., Burke-Spolaor, S., Deller, A.T. et al. (2014). Real-time adaptive event detection in astronomical data streams. IEEE Intell. Syst. 29 48–55.
  • [33] Tsybakov, A.B. (2009). Introduction to Nonparametric Estimation. Mathematics & Applications 41. Berlin: Springer.
  • [34] Wald, A. (1945). Sequential tests of statistical hypotheses. Ann. Math. Stat. 16 117–186.
  • [35] Wang, H., Tang, M., Park, Y. and Priebe, C.E. (2014). Locality statistics for anomaly detection in time series of graphs. IEEE Trans. Signal Process. 62 703–717.
  • [36] Zhao, Q. and Ye, J. (2010). Quickest detection in multiple on–off processes. IEEE Trans. Signal Process. 58 5994–6006.
  • [37] Zhu, K. and Ying, L. (2013). Information source detection in the SIR model: A sample path based approach. In Information Theory and Applications Workshop (ITA) 1–9.
  • [38] Zigangirov, K.Š. (1966). On a problem of optimal scanning. Theory Probab. Appl. 11 294–298.