## Electronic Journal of Statistics

### Lower bound in regression for functional data by representation of small ball probabilities

André Mas

#### Abstract

The minimax rate for estimating the regression function $r(\cdot)=\mathbb{E}(y|X=\cdot)$ when $y\in\mathbb{R}$ and $X$ takes values in a function space is the initial motivation of this work. Recent articles underline the major role of the shifted small ball probability $\mathbb{P}(\Vert X-x_{0}\Vert <\cdot)$ in the variance of classical estimates. The main results are twofold. First, starting from a theorem by [41], we study the small ball probability $\mathbb{P}(S<\varepsilon)$ when $\varepsilon\downarrow0$ with $S=\sum_{i=1}^{+\infty}\lambda_{i}Z_{i}$ where the $Z_{i}$’s are i.i.d. positive and $(\lambda_{i})_{i\in\mathbb{N}}$ a positive nonincreasing sequence such that $\sum\lambda_{i}<+\infty$. It is shown that $\mathbb{P}(S<\cdot)$ belongs to a class of functions introduced by de Haan, well-known in extreme value theory, the class of Gamma-varying functions, for which an exponential-integral representation is available. Second this approach allows to derive minimax lower bounds for the risk at a fixed point $x_{0}$ when $X\in\mathcal{H}$ some Hilbert space of functions. Denoting this minimax risk: $\mathcal{R}_{n}^{\ast}=\inf_{T_{n}}\sup_{r\in\mathcal{E}}\mathbb{E}\left\vert T_{n}-r(x_{0})\right\vert ^{2}$ where $T_{n}$ is any estimate of $r(x_{0})$ and $\mathcal{E}$ is some class of smooth functions from $\mathcal{H}$ to $\mathbb{R}$ it turns out that, in a general framework, $n^{\tau}\mathcal{R}_{n}^{\ast}\rightarrow+\infty$ for any $\tau>0.$ This negative result may pave the way towards new approaches for modeling regression with functional data.

#### Article information

Source
Electron. J. Statist., Volume 6 (2012), 1745-1778.

Dates
First available in Project Euclid: 27 September 2012

https://projecteuclid.org/euclid.ejs/1348753351

Digital Object Identifier
doi:10.1214/12-EJS726

Mathematical Reviews number (MathSciNet)
MR2988463

Zentralblatt MATH identifier
1295.62043

#### Citation

Mas, André. Lower bound in regression for functional data by representation of small ball probabilities. Electron. J. Statist. 6 (2012), 1745--1778. doi:10.1214/12-EJS726. https://projecteuclid.org/euclid.ejs/1348753351

#### References

• [1] de Acosta A. (1983): Small deviations in the functional central limit theorem with applications to functional laws of the iterated logarithm, Ann. Probab., 11, 78–101.
• [2] Ait-Saadi A., Ferraty F., Kassa R., Vieu P. (2008): Cross-validated estimations in the single-functional index model, Statistics, 42, 475–494.
• [3] Aurzada F., Lifshits, M., Linde W. (2009): Small deviations of stable processes and entropy of associated random operators, Bernoulli, 15, 1305–1334.
• [4] Berlinet A., Elamine A., Mas A. (2012): Local linear regression for functional data, Ann. Instit. Statist. Math., 63, 1047–1075.
• [5] Bingham N.H., Goldie C.M. and Teugels J.L. (1987):, Regular Variations. Encyclopedia of Mathematics and Its Applications, Cambridge University Press.
• [6] Bloom S. (1976): A characterization of B-slowly varying functions, Proc. Amer. Math. Soc., 54, 243–250.
• [7] Borell C. (1976): Gaussian Radon measures on locally convex spaces, Math. Scand., 38, 265–285.
• [8] Cai T., Hall P. (2006): Prediction in functional linear regression, Ann. Statist., 34, 2159–2179.
• [9] Cardot H., Johannes J. (2010): Thresholding projection estimators in functional linear models, J. Multivariate Anal., 101, 395–408.
• [10] Crambes C., Kneip A., Sarda P. (2009): Smoothing splines estimators for functional linear regression, Ann. Statist., 37, 35–72.
• [11] Dabo-Niang S. (2002): Estimation de la densité dans un espace de dimension infinie: Application aux diffusions, C. R. Math. Acad. Sci. Paris., 334, 213–216.
• [12] Delaigle A., Hall P. (2010): Defining probability for a distribution of random functions, Ann. Statist., 38, 1171–1193.
• [13] Dembo A., Meyer-Wolf E., Zeitouni O. (1995): Exact behavior of Gaussian semi-norms, Statist. Probab. Lett., 23, 275–280.
• [14] Dunker T., Lifshits M.A., Linde W. (1998): Small deviation probabilities of sums of independent random variables. In High Dimensional Probability (Oberwolfach, 1996), 59–74, Progr. Probab., 43, Birkhäuser, Basel.
• [15] Erickson K. (1980): Rates of escape of infinite-dimensional Brownian motion, Ann. Probab., 8, 325–338.
• [16] Fan, J. (1991): On the optimal rates of convergence for nonparametric deconvolution problems., Ann. Statist., 19, 1257–1272.
• [17] Ferraty F., Hall P., Vieu P. (2010): Most-predictive design points for functional data predictors, Biometrika., 97 n$^\circ$34, 807–824.
• [18] Ferraty F., Laksaci A., Tadj A., Vieu P. (2010): Rate of uniform consistency for nonparametric estimates with functional variables., J. Statist. Plann. Inference, 140, no. 2, 335–352.
• [19] Ferraty F., Mas A., Vieu P. (2007): Advances on nonparametric regression from functional data, Aust. New Zealand J. Stat., 49 n$^\circ$3, 1–20.
• [20] Ferraty F., van Keilegom I., Vieu P. (2010): On the validity of the boostrap in the nonparametric functional regression, Scand. J. Stat., 37, 286–306.
• [21] Ferraty F., Vieu P. (2004): Nonparametric models for functional data, with applications in regression, time series prediction and curve discrimination., J. Nonparametr. Stat., 16, 111–125.
• [22] Ferraty F., Vieu P. (2006):, Nonparametric Functional Data Analysis: Theory and Practice. Springer, Berlin.
• [23] Ferraty F., Vieu P. (2009): Additive prediction and bosting for functional data, Comput. Statist. Data Anal., 53, 1400–1413.
• [24] Ferraty F., Vieu P. (2010): Kernel regression estimation fur functional data. In, The Oxford Handbook of Functional data Analysis, Ferraty and Romain Eds, 72–129.
• [25] Fisher R.A., Tippett L.H.C. (1928): Limiting forms of the frequency distribution of the largest or smaller member of a sample, Proceedings of the Cambridge Philosophical Society, 24, 180–190.
• [26] Gaiffas S. (2005): Convergence rates for pointwise curve estimation with a degenerate design, Math. Methods Statist., 14, 1–27.
• [27] Gardes L., Girard S., Lekina A. (2010): Functional nonparametric estimation of conditional extremal quantiles, J. Multivariate Anal., 101, 419–433.
• [28] Gnedenko B.V. (1943): Sur la distribution limite du terme maximum d’une série aléatoire, Ann. of Math., 44, 423–453.
• [29] de Haan L. (1971): A form of regular variation and its application to the domain of attraction of the double exponential distribution, Z. Wahrscheinlichkeitstheorie. verw. Geb., 17, 241–258.
• [30] de Haan L. (1974): Equivalence classes of regularly varying functions, Stochastic Process. Appl., 2, 243–259.
• [31] Hall P., Heckman N. (2002): Estimating and depicting the structure of a distribution of random functions, Biometrika, 89, 145–158.
• [32] Hall P., Horowitz J.L.N. (2007): Methodology and convergence rate for functional linear regression., Ann. Statist., 35, 70–91.
• [33] Jurek Z.J. (1979): A representation of the characteristic functions of stable distributions in Hilbert spaces., Banach Center Publications, 5 PWN Warsaw, 151–158.
• [34] Kuelbs J., Li W.V., Linde W. (1994): Lim inf results for Gaussian samples and Chung’s functional LIL, Ann. Probab., 22, 1879–1903.
• [35] Ledoux M., Talagrand M. (1991):, Probability in Banach Spaces. Isoperimetry and Processes, Ergebnisse der Mathematik und ihrer Grenzgebiete, 23, Springer-Verlag.
• [36] Li W.V., Linde W. (1993): Small ball problems for non centered Gaussian measures, Probab. Math. Statist., 14, 231–251.
• [37] Li W.V., Linde W. (1999): Approximation, metric entropy and small ball estimates for Gaussian measures., Ann. Probab., 27, 1556–1578.
• [38] Li W.V., Linde W. (2004): Small deviations of stable processes via metric entropy, J. Theoret. Probab., 17, 261–284.
• [39] Li, W.V., Shao Q.-M. (2001): Gaussian processes: Inequalities, small ball probabilities and applications, In Rao and Shanbhag (eds.), Handbook of Statistics, 19, Elsevier, New-York, 533–597.
• [40] Lifshits M.A. (1995): Gaussian random functions. Mathematics and its Applications, 322. Kluwer Academic Publishers, Dordrecht.
• [41] Lifshits M.A. (1997): On the lower tail probabilities of some random series, Ann. Probab., 25, 424–442.
• [42] Mas A. (2008): Local functional principal component analysis, Complex Analysis and Operator Theory, 2, 135–167.
• [43] Meyer-Wolf E., Zeitouni O. (1993): The probability of small Gaussian ellipsoïds, Ann. Probab., 21, 14–24.
• [44] Müller H.G., Yan X. (2001): On local moments, J. Multivariate Anal., 76, 90–109.
• [45] Müller H.G., Yao F. (2008): Functional additive models, J. Amer. Statist. Assoc., 103, 1534–1544.
• [46] Rachdi M., Vieu P. (2007): Nonparametric regression for functional data: Automatic smoothing parameter selection, Journal of Statistical Planning and Inference, 137, 2784–2801.
• [47] Ramsay, J.O. and Silverman, B.W. (2005):, Functional data analysis (Second Ed.). Springer.
• [48] Skorohod, A.V. (1967): On the densities of probability measures in functional spaces. Proc. Fifth Berkeley Sympos. Math. Statist. and Probability (Berkeley, Calif., 1965/66) Vol. II: Contributions to Probability Theory, Part 1, p. 163–182 Univ. California Press, Berkeley, Calif.
• [49] Stone C.J. (1980): Optimal rates of convergence for nonparametric estimators, Ann. Statist., 8, 1348–1360.
• [50] Stone C.J. (1982): Optimal global rates of convergence for nonparametric regression., Ann. Statist., 10, 1040–1053.
• [51] Stone C.J. (1985): Additive regression and other nonparametric models, Ann. Statist., 13, 689–705.
• [52] Sytaya G.N. (1974): On some asymptotic representations of the Gaussian measure in a Hilbert space, Theory of Stochastic Processes, 2, 93–104.
• [53] Talagrand M. (1992): On the rate of convergence in Strassen’s LIL, Progr. Probab., Birkhauser, Boston, 339–351.
• [54] Tsybakov A. (2004): Introduction à l’estimation non-paramétrique. Mathématiques et applications., Springer.
• [55] van der Vaart, A.W., van Zanten, J.H. (2007): Bayesian inference with rescaled Gaussian process priors, Electronic J. Statist., 1, 433–448.
• [56] van der Vaart, A.W., van Zanten, J.H. (2008): Rates of contraction of posterior distributions based on Gaussian process priors., Ann. Statist., 36, 1435–1463.