The Annals of Statistics

Spatially-adaptive sensing in nonparametric regression

Adam D. Bull

Full-text: Open access


While adaptive sensing has provided improved rates of convergence in sparse regression and classification, results in nonparametric regression have so far been restricted to quite specific classes of functions. In this paper, we describe an adaptive-sensing algorithm which is applicable to general nonparametric-regression problems. The algorithm is spatially adaptive, and achieves improved rates of convergence over spatially inhomogeneous functions. Over standard function classes, it likewise retains the spatial adaptivity properties of a uniform design.

Article information

Ann. Statist., Volume 41, Number 1 (2013), 41-62.

First available in Project Euclid: 5 March 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression
Secondary: 62L05: Sequential design 62G20: Asymptotic properties

Nonparametric regression adaptive sensing sequential design active learning spatial adaptation spatially inhomogeneous functions


Bull, Adam D. Spatially-adaptive sensing in nonparametric regression. Ann. Statist. 41 (2013), no. 1, 41--62. doi:10.1214/12-AOS1064.

Export citation


  • Arias-Castro, E., Candes, E. J. and Davenport, M. (2011). On the fundamental limits of adaptive sensing. Available at arXiv:1111.4646.
  • Beygelzimer, A., Dasgupta, S. and Langford, J. (2009). Importance weighted active learning. In Proceedings of the 26th Annual International Conference on Machine Learning 49–56. Omnipress, Madison, WI.
  • Boufounos, P., Cevher, V., Gilbert, A., Li, Y. and Strauss, M. (2012). What’s the frequency, Kenneth? Sublinear Fourier sampling off the grid. In 15th International Workshop, APPROX 2012, and 16th International Workshop, RANDOM 2012, Cambridge, MA, USA, August 1517, 2012, Proceedings. Springer, New York.
  • Bull, A. D. (2012). Honest adaptive confidence bands and self-similar functions. Electron. J. Stat. 6 1490–1516.
  • Bull, A. D. (2013). Supplement to “Spatially-adaptive sensing in nonparametric regression.” DOI:10.1214/12-AOS1064SUPP.
  • Castro, R. M. (2012). Adaptive sensing performance lower bounds for sparse signal estimation and testing. Available at arXiv:1206.0648.
  • Castro, R. M. and Nowak, R. D. (2008). Minimax bounds for active learning. IEEE Trans. Inform. Theory 54 2339–2353.
  • Castro, R., Willett, R. and Nowak, R. (2006). Faster rates in regression via active learning. Adv. Neural Inf. Process. Syst. 18 179.
  • Cohen, A., Daubechies, I. and Vial, P. (1993). Wavelets on the interval and fast wavelet transforms. Appl. Comput. Harmon. Anal. 1 54–81.
  • Cohn, D., Atlas, L. and Ladner, R. (1994). Improving generalization with active learning. Machine Learning 15 201–221.
  • Cohn, D. A., Ghahramani, Z. and Jordan, M. I. (1996). Active learning with statistical models. J. Artificial Intelligence Res. 4 129–145.
  • Davenport, M. A. and Arias-Castro, E. (2012). Compressive binary search. Available at arXiv:1202.0937.
  • Donoho, D. L. and Johnstone, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 425–455.
  • Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879–921.
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G. and Picard, D. (1995). Wavelet shrinkage: Asymptopia? J. R. Stat. Soc. Ser. B Stat. Methodol. 57 301–369.
  • Efromovich, S. (2008). Optimal sequential design in a controlled non-parametric regression. Scand. J. Stat. 35 266–285.
  • Fan, J. and Gijbels, I. (1995). Data-driven bandwidth selection in local polynomial fitting: Variable bandwidth and spatial adaptation. J. R. Stat. Soc. Ser. B Stat. Methodol. 57 371–394.
  • Fan, J. and Lv, J. (2008). Sure independence screening for ultrahigh dimensional feature space. J. R. Stat. Soc. Ser. B Stat. Methodol. 70 849–911.
  • Fan, J., Hall, P., Martin, M. and Patil, P. (1999). Adaptation to high spatial inhomogeneity using wavelet methods. Statist. Sinica 9 85–102.
  • Faraway, J. J. (1990). Sequential design for the nonparametric regression of curves and surfaces. In Computing Science and Statistics 22 104–110. Interface, Fairfax Station, VA.
  • Frisch, U. and Parisi, G. (1985). On the singularity structure of fully developed turbluence. In Turbulence and Predictability in Geophysical Fluid Dynamics and Climate Dynamics 84-7. Soc. Italiana di Fisica, Bologna, Italy.
  • Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122–1170.
  • Hall, P. and Molchanov, I. (2003). Sequential methods for design-adaptive estimation of discontinuities in regression curves and surfaces. Ann. Statist. 31 921–941.
  • Hanneke, S. (2011). Rates of convergence in active learning. Ann. Statist. 39 333–361.
  • Härdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. (1998). Wavelets, Approximation, and Statistical Applications. Lecture Notes in Statistics 129. Springer, New York.
  • Haupt, J., Castro, R. M. and Nowak, R. (2011). Distilled sensing: Adaptive sampling for sparse detection and estimation. IEEE Trans. Inform. Theory 57 6222–6235.
  • Iwen, M. A. (2009). Group testing strategies for recovery of sparse signals in noise. In Conference on Signals, Systems and Computers, 2009 Conference Record of the Forty-Third Asilomar 1561–1565. IEEE, New York.
  • Jaffard, S. (2000). On the Frisch–Parisi conjecture. J. Math. Pures Appl. (9) 79 525–552.
  • Kerkyacharian, G. and Picard, D. (2004). Regression in random design and warped wavelets. Bernoulli 10 1053–1105.
  • Koltchinskii, V. (2010). Rademacher complexities and bounding the excess risk in active learning. J. Mach. Learn. Res. 11 2457–2485.
  • Lepski, O. V., Mammen, E. and Spokoiny, V. G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors. Ann. Statist. 25 929–947.
  • Malloy, M. and Nowak, R. (2011a). On the limits of sequential testing in high dimensions. Available at arXiv:1105.4540.
  • Malloy, M. and Nowak, R. (2011b). Sequential analysis in high dimensional multiple testing and sparse recovery. Available at arXiv:1103.5991.
  • Meinshausen, N. and Bühlmann, P. (2010). Stability selection. J. R. Stat. Soc. Ser. B Stat. Methodol. 72 417–473.
  • Nason, G. (2010). Wavethresh: Wavelet statistics and transforms. R package version 4.5.
  • Picard, D. and Tribouley, K. (2000). Adaptive confidence interval for pointwise curve estimation. Ann. Statist. 28 298–335.
  • van de Geer, S. A. and Bühlmann, P. (2009). On the conditions used to prove oracle results for the Lasso. Electron. J. Stat. 3 1360–1392.

Supplemental materials