## The Annals of Statistics

### Online estimation of the geometric median in Hilbert spaces: Nonasymptotic confidence balls

#### Abstract

Estimation procedures based on recursive algorithms are interesting and powerful techniques that are able to deal rapidly with very large samples of high dimensional data. The collected data may be contaminated by noise so that robust location indicators, such as the geometric median, may be preferred to the mean. In this context, an estimator of the geometric median based on a fast and efficient averaged nonlinear stochastic gradient algorithm has been developed by [Bernoulli 19 (2013) 18–43]. This work aims at studying more precisely the nonasymptotic behavior of this nonlinear algorithm by giving nonasymptotic confidence balls in general separable Hilbert spaces. This new result is based on the derivation of improved $L^{2}$ rates of convergence as well as an exponential inequality for the nearly martingale terms of the recursive nonlinear Robbins–Monro algorithm.

#### Article information

Source
Ann. Statist., Volume 45, Number 2 (2017), 591-614.

Dates
Revised: February 2016
First available in Project Euclid: 16 May 2017

https://projecteuclid.org/euclid.aos/1494921951

Digital Object Identifier
doi:10.1214/16-AOS1460

Mathematical Reviews number (MathSciNet)
MR3650394

Zentralblatt MATH identifier
1371.62027

Subjects
Primary: 62G05: Estimation
Secondary: 62L20: Stochastic approximation

#### Citation

Cardot, Hervé; Cénac, Peggy; Godichon-Baggioni, Antoine. Online estimation of the geometric median in Hilbert spaces: Nonasymptotic confidence balls. Ann. Statist. 45 (2017), no. 2, 591--614. doi:10.1214/16-AOS1460. https://projecteuclid.org/euclid.aos/1494921951

#### References

• [1] Arnaudon, M., Dombry, C., Phan, A. and Yang, L. (2012). Stochastic algorithms for computing means of probability measures. Stochastic Process. Appl. 122 1437–1455.
• [2] Bach, F. (2014). Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression. J. Mach. Learn. Res. 15 595–627.
• [3] Bach, F. and Moulines, E. (2013). Non-strongly-convex smooth stochastic approximation with convergence rate $o(1/n)$. In Advances in Neural Information Processing Systems 773–781. Curran Associates, Inc., Red Hook, NY.
• [4] Bali, J. L., Boente, G., Tyler, D. E. and Wang, J.-L. (2011). Robust functional principal components: A projection-pursuit approach. Ann. Statist. 39 2852–2882.
• [5] Balsubramani, A., Dasgupta, S. and Freund, Y. (2013). The fast convergence of incremental PCA. In Advances in Neural Information Processing Systems 3174–3182. Curran Associates, Inc., Red Hook, NY.
• [6] Beck, A. and Sabach, S. (2015). Weiszfeld’s method: Old and new results. J. Optim. Theory Appl. 164 1–40.
• [7] Cardot, H., Cénac, P. and Godichon-Baggioni, A. (2016). Supplement to “Online estimation of the geometric median in Hilbert spaces: Nonasymptotic confidence balls.” DOI:10.1214/16-AOS1460SUPP.
• [8] Cardot, H., Cénac, P. and Zitt, P.-A. (2013). Efficient and fast estimation of the geometric median in Hilbert spaces with an averaged stochastic gradient algorithm. Bernoulli 19 18–43.
• [9] Chakraborty, A. and Chaudhuri, P. (2014). The spatial distribution in infinite dimensional spaces and related quantiles and depths. Ann. Statist. 42 1203–1231.
• [10] Chaudhuri, P. (1992). Multivariate location estimation using extension of $R$-estimates through $U$-statistics type approach. Ann. Statist. 20 897–916.
• [11] Fletcher, P. T., Venkatasubramanian, S. and Joshi, S. (2009). The geometric median on Riemannian manifolds with application to robust atlas estimation. NeuroImage 45 S143–S152.
• [12] Gervini, D. (2008). Robust functional estimation using the median and spherical principal components. Biometrika 95 587–600.
• [13] Haldane, J. B. S. (1948). Note on the median of a multivariate distribution. Biometrika 35 414–417.
• [14] Kemperman, J. H. B. (1987). The median of a finite measure on a Banach space. In Statistical Data Analysis Based on the $L_{1}$-Norm and Related Methods (Neuchâtel, 1987) 217–230. North-Holland, Amsterdam.
• [15] Kraus, D. and Panaretos, V. M. (2012). Dispersion operators and resistant second-order functional data analysis. Biometrika 99 813–832.
• [16] Kuhn, H. W. (1973). A note on Fermat’s problem. Math. Program. 4 98–107.
• [17] Locantore, N., Marron, J. S., Simpson, D. G., Tripoli, N., Zhang, J. T. and Cohen, K. L. (1999). Robust principal component analysis for functional data. TEST 8 1–73.
• [18] Minsker, S. (2015). Geometric median and robust estimation in Banach spaces. Bernoulli 21 2308–2335.
• [19] Minsker, S., Srivastava, S., Lin, L. and Dunson, D. (2014). Scalable and robust Bayesian inference via the median posterior. In Proceedings of the 31st International Conference on Machine Learning (ICML-14) 1656–1664.
• [20] Möttönen, J., Nordhausen, K. and Oja, H. (2010). Asymptotic theory of the spatial median. In Nonparametrics and Robustness in Modern Statistical Inference and Time Series Analysis: A Festschrift in Honor of Professor Jana Jurečková. Inst. Math. Stat. Collect. 7 182–193. IMS, Beachwood, OH.
• [21] Pelletier, M. (2000). Asymptotic almost sure efficiency of averaged stochastic algorithms. SIAM J. Control Optim. 39 49–72 (electronic).
• [22] Pinelis, I. (1994). Optimum bounds for the distributions of martingales in Banach spaces. Ann. Probab. 22 1679–1706.
• [23] Polyak, B. T. and Juditsky, A. B. (1992). Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30 838–855.
• [24] Rudelson, M. and Vershynin, R. (2010). Non-asymptotic theory of random matrices: Extreme singular values. In Proceedings of the International Congress of Mathematicians. Volume III 1576–1602. Hindustan Book Agency, New Delhi.
• [25] Small, C. G. (1990). A survey of multidimensional medians. Int. Stat. Rev. 58 263–277.
• [26] Tarrès, P. and Yao, Y. (2014). Online learning as stochastic approximation of regularization paths: Optimality and almost-sure convergence. IEEE Trans. Inform. Theory 60 5716–5735.
• [27] Vardi, Y. and Zhang, C.-H. (2000). The multivariate $L_{1}$-median and associated data depth. Proc. Natl. Acad. Sci. USA 97 1423–1426 (electronic).
• [28] Weiszfeld, E. (1937). Sur le point pour lequel la somme des distances de $n$ points donnés est minimum. Tohoku Math. J. (2) 43 355–386.
• [29] Woodroofe, M. (1972). Normal approximation and large deviations for the Robbins–Monro process. Z. Wahrsch. Verw. Gebiete 21 329–338.

#### Supplemental materials

• Supplement to “Online estimation of the geometric median in Hilbert spaces: Nonasymptotic confidence balls”. We provide the proofs of some technical ancillary lemmas and propositions.