Bernoulli

  • Bernoulli
  • Volume 16, Number 4 (2010), 1064-1085.

A Bernstein-type inequality for suprema of random processes with applications to model selection in non-Gaussian regression

Yannick Baraud

Full-text: Open access

Abstract

Let $(X_t)_{t∈T}$ be a family of real-valued centered random variables indexed by a countable set $T$. In the first part of this paper, we establish exponential bounds for the deviation probabilities of the supremum $Z = \sup_{t∈T} X_t$ by using the generic chaining device introduced in Talagrand (Inst. Hautes Études Sci. Publ. Math. 81 (1995) 73–205). Compared to concentration-type inequalities, these bounds offer the advantage of holding under weaker conditions on the family $(X_t)_{t∈T}$. The second part of the paper is oriented toward statistics. We consider the regression setting $Y = f + ξ$, where $f$ is an unknown vector in $ℝ^n$ and $ξ$ is a random vector, the components of which are independent, centered and admit finite Laplace transforms in a neighborhood of $0$. Our aim is to estimate $f$ from the observation of $Y$ by means of a model selection approach among a collection of linear subspaces of $ℝ^n$. The selection procedure we propose is based on the minimization of a penalized criterion, the penalty of which is calibrated by using the deviation bounds established in the first part of this paper. More precisely, we study suprema of random variables of the form $X_t = ∑_{i=1}^n t_iξ_i$, where $t$ varies in the unit ball of a linear subspace of $ℝ^n$. Finally, we show that our estimator satisfies an oracle-type inequality under suitable assumptions on the metric structures of the linear spaces of the collection.

Article information

Source
Bernoulli, Volume 16, Number 4 (2010), 1064-1085.

Dates
First available in Project Euclid: 18 November 2010

Permanent link to this document
https://projecteuclid.org/euclid.bj/1290092896

Digital Object Identifier
doi:10.3150/09-BEJ245

Mathematical Reviews number (MathSciNet)
MR2759169

Zentralblatt MATH identifier
05858609

Keywords
Bernstein’s inequality model selection regression supremum of a random process

Citation

Baraud, Yannick. A Bernstein-type inequality for suprema of random processes with applications to model selection in non-Gaussian regression. Bernoulli 16 (2010), no. 4, 1064--1085. doi:10.3150/09-BEJ245. https://projecteuclid.org/euclid.bj/1290092896


Export citation

References

  • Baraud, Y. (2000). Model selection for regression on a fixed design. Probab. Theory Related Fields 117 467–493.
  • Baraud, Y. (2009). A Bernstein-type inequality for suprema of random processes with applications to model selection in non-Gaussian regression. Technical Report. Available at arXiv:0909.1863v1.
  • Baraud, Y., Comte, F. and Viennet, G. (2001). Model selection for (auto-)regression with dependent data. ESAIM Probab. Statist. 5 33–49 (electronic).
  • Barron, A., Birgé, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301–413.
  • Barron, A.R. and Cover, T.M. (1991). Minimum complexity density estimation. IEEE Trans. Inform. Theory 37 1034–1054.
  • Birgé, L. (1983). Approximation dans les espaces métriques et théorie de l’estimation. Z. Wahrsch. Verw. Gebiete 65 181–237.
  • Birgé, L. and Massart, P. (2001). Gaussian model selection. J. Eur. Math. Soc. (JEMS) 3 203–268.
  • Bousquet, O. (2002). A Bennett concentration inequality and its application to suprema of empirical processes. C. R. Math. Acad. Sci. Paris 334 495–500.
  • Bousquet, O. (2003). Concentration inequalities for sub-additive functions using the entropy method. In Stochastic Inequalities and Applications. Progr. Probab. 56 213–247. Basel: Birkhäuser.
  • Klartag, B. and Mendelson, S. (2005). Empirical processes and random projections. J. Funct. Anal. 225 229–245.
  • Klein, T. and Rio, E. (2005). Concentration around the mean for maxima of empirical processes. Ann. Probab. 33 1060–1077.
  • Ledoux, M. (1996). On Talagrand’s deviation inequalities for product measures. ESAIM Probab. Statist. 1 63–87 (electronic).
  • Massart, P. (2000). About the constants in Talagrand’s concentration inequalities for empirical processes. Ann. Probab. 28 863–884.
  • Massart, P. (2007). Concentration Inequalities and Model Selection. Lectures from the 33rd Summer School on Probability Theory Held in Saint-Flour, July 6–23, 2003. Lecture Notes in Math. 1896. Berlin: Springer.
  • Mendelson, S. (2008). On weakly bounded empirical processes. Math. Ann. 340 293–314.
  • Mendelson, S., Pajor, A. and Tomczak-Jaegermann, N. (2007). Reconstruction and subgaussian operators in asymptotic geometric analysis. Geom. Funct. Anal. 17 1248–1282.
  • Rio, E. (2002). Une inégalité de Bennett pour les maxima de processus empiriques. Ann. Inst. H. Poincaré Probab. Statist. 38 1053–1057.
  • Sauvé, M. (2008). Histogram selection in non gaussian regression. ESAIM Probab. Statist. 13 70–86.
  • Sudakov, V.N. and Cirel’son, B.S. (1974). Extremal properties of half-spaces for spherically invariant measures. Problems in the theory of probability distributions, II. Zap. Naučn. Sem. Leningrad. Otdel. Mat. Inst. Steklov. (LOMI) 41 14–24, 165.
  • Talagrand, M. (1995). Concentration of measure and isoperimetric inequalities in product spaces. Inst. Hautes Études Sci. Publ. Math. 81 73–205.
  • Talagrand, M. (2005). The Generic Chaining: Upper and Lower Bounds of Stochastic Processes. Springer Monographs in Mathematics. Berlin: Springer.
  • van de Geer, S. (1990). Estimating a regression function. Ann. Statist. 18 907–924.