Electronic Journal of Statistics

Optimal-order bounds on the rate of convergence to normality in the multivariate delta method

Iosif Pinelis and Raymond Molzon

Full-text: Open access

Abstract

Uniform and nonuniform Berry–Esseen (BE) bounds of optimal orders on the rate of convergence to normality in the delta method for vector statistics are obtained. The results are applicable almost as widely as the delta method itself – except that, quite naturally, the order of the moments needed to be finite is generally $3/2$ times as large as that for the corresponding central limit theorems. Our BE bounds appear new even for the one-dimensional delta method, that is, for smooth functions of the sample mean of univariate random variables. Specific applications to Pearson’s, noncentral Student’s and Hotelling’s statistics, sphericity test statistics, a regularized canonical correlation, and maximum likelihood estimators (MLEs) are given; all these uniform and nonuniform BE bounds appear to be the first known results of these kinds, except for uniform BE bounds for MLEs. The new method allows one to obtain bounds with explicit and rather moderate-size constants. For instance, one has the uniform BE bound $3.61\mathbb{E}(Y_{1}^{6}+Z_{1}^{6})\,(1+\sigma^{-3})/\sqrt{n}$ for the Pearson sample correlation coefficient based on independent identically distributed random pairs $(Y_{1},Z_{1}),\dots,(Y_{n},Z_{n})$ with $\mathbb{E} Y_{1}=\mathbb{E}Z_{1}=\mathbb{E}Y_{1}Z_{1}=0$ and $\mathbb{E}Y_{1}^{2}=\mathbb{E}Z_{1}^{2}=1$, where $\sigma:=\sqrt{\mathbb{E}Y_{1}^{2}Z_{1}^{2}}$.

Article information

Source
Electron. J. Statist., Volume 10, Number 1 (2016), 1001-1063.

Dates
Received: August 2015
First available in Project Euclid: 12 April 2016

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1460463653

Digital Object Identifier
doi:10.1214/16-EJS1133

Mathematical Reviews number (MathSciNet)
MR3486424

Zentralblatt MATH identifier
1337.60024

Subjects
Primary: 60F05: Central limit and other weak theorems 60E15: Inequalities; stochastic orderings 62F12: Asymptotic properties of estimators
Secondary: 60E10: Characteristic functions; other transforms 62F03: Hypothesis testing 62F05: Asymptotic properties of tests 62G10: Hypothesis testing 62G20: Asymptotic properties

Keywords
Berry–Esseen bound canonical correlation delta method rates of convergence Cramér’s tilt exponential inequalities maximum likelihood estimators noncentral Hotelling’s statistic noncentral Student’s statistic nonlinear statistics Pearson’s correlation coefficient sphericity test

Citation

Pinelis, Iosif; Molzon, Raymond. Optimal-order bounds on the rate of convergence to normality in the multivariate delta method. Electron. J. Statist. 10 (2016), no. 1, 1001--1063. doi:10.1214/16-EJS1133. https://projecteuclid.org/euclid.ejs/1460463653


Export citation

References

  • [1] Anastasiou, A. and Ley, C. (2015). New simpler bounds to assess the asymptotic normality of the maximum likelihood estimator., http://arxiv.org/abs/1508.04948.
  • [2] Anastasiou, A. and Reinert, G. (2015). Bounds for the normal approximation of the maximum likelihood estimator., http://arxiv.org/abs/1411.2391.
  • [3] Bennett, G. (1962). Probability inequalities for the sum of independent random variables., J. Amer. Statist. Assoc. 57 33–45.
  • [4] Bentkus, V., Bloznelis, M. and Götze, F. (1996). A Berry-Esséen bound for Student’s statistic in the non-i.i.d. case., J. Theoret. Probab. 9 765–796.
  • [5] Bentkus, V. and Götze, F. (1993). On smoothness conditions and convergence rates in the CLT in Banach spaces., Probab. Theory Related Fields 96 137–151.
  • [6] Bentkus, V. and Götze, F. (1996). The Berry-Esseen bound for Student’s statistic., Ann. Probab. 24 491–503.
  • [7] Bentkus, V., Jing, B.-Y., Shao, Q.-M. and Zhou, W. (2007). Limiting distributions of the non-central $t$-statistic and their applications to the power of $t$-tests under non-normality., Bernoulli 13 346–364.
  • [8] Bhattacharya, R. N. and Ghosh, J. K. (1978). On the validity of the formal Edgeworth expansion., Ann. Statist. 6 434–451.
  • [9] Bolthausen, E. and Götze, F. (1993). The rate of convergence for multivariate sampling statistics., Ann. Statist. 21 1692–1710.
  • [10] Chen, L. H. Y. and Fang, X. (2011). Multivariate normal approximation by Stein’s method: the concentration inequality approach (preprint)., http://arxiv.org/abs/1111.4073.
  • [11] Chen, L. H. Y. and Shao, Q.-M. (2007). Normal approximation for nonlinear statistics using a concentration inequality approach., Bernoulli 13 581–599.
  • [12] Chen, S. X., Zhang, L.-X. and Zhong, P.-S. (2010). Tests for high-dimensional covariance matrices., J. Amer. Statist. Assoc. 105 810–819.
  • [13] Cupidon, J., Eubank, R., Gilliam, D. and Ruymgaart, F. (2008). Some properties of canonical correlations and variates in infinite dimensions., J. Multivariate Anal. 99 1083–1104.
  • [14] Cupidon, J., Gilliam, D. S., Eubank, R. and Ruymgaart, F. (2007). The delta method for analytic functions of random operators with application to functional data., Bernoulli 13 1179–1194.
  • [15] de Acosta, A. and Samur, J. D. (1979). Infinitely divisible probability measures and the converse Kolmogorov inequality in Banach spaces., Studia Math. 66 143–160.
  • [16] Dowson, D. C. and Landau, B. V. (1982). The Fréchet distance between multivariate normal distributions., J. Multivariate Anal. 12 450–455.
  • [17] Eubank, R. L. and Hsing, T. (2008). Canonical correlation for stochastic processes., Stochastic Process. Appl. 118 1634–1661.
  • [18] Fisher, T. J., Sun, X. and Gallagher, C. M. (2010). A new test for sphericity of the covariance matrix for high dimensional data., J. Multivariate Anal. 101 2554–2570.
  • [19] Gaines, G., Kaphle, K. and Ruymgaart, F. (2011). Application of a delta-method for random operators to testing equality of two covariance operators., Math. Methods Statist. 20 232–245.
  • [20] Gamboa, F., Janon, A., Klein, T., Lagnoux-Renaudie, A. and Prieur, C. (2013). Statistical inference for Sobol pick freeze Monte Carlo method (preprint)., http://arxiv.org/pdf/1303.6447.pdf.
  • [21] Gilliam, D. S., Hohage, T., Ji, X. and Ruymgaart, F. (2009). The Fréchet derivative of an analytic function of a bounded operator with some applications., Int. J. Math. Math. Sci. Art. ID 239025, 17.
  • [22] Giné, E., Götze, F. and Mason, D. M. (1997). When is the Student $t$-statistic asymptotically standard normal?, Ann. Probab. 25 1514–1531.
  • [23] Givens, C. R. and Shortt, R. M. (1984). A class of Wasserstein metrics for probability distributions., Michigan Math. J. 31 231–240.
  • [24] Götze, F. (1986). On the rate of convergence in the central limit theorem in Banach spaces., Ann. Probab. 14 922–942.
  • [25] Götze, F. (1991). On the rate of convergence in the multivariate CLT., Ann. Probab. 19 724–739.
  • [26] Hall, P. and Wang, Q. (2004). Exact convergence rate and leading term in central limit theorem for Student’s $t$ statistic., Ann. Probab. 32 1419–1437.
  • [27] He, G., Müller, H.-G. and Wang, J.-L. (2004). Methods of canonical analysis for functional data., J. Statist. Plann. Inference 122 141–159. Contemporary data analysis: theory and methods.
  • [28] Heyde, C. C. (1997)., Quasi-likelihood and its application. Springer Series in Statistics. Springer-Verlag, New York A general approach to optimal parameter estimation.
  • [29] Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables., J. Amer. Statist. Assoc. 58 13–30.
  • [30] Hoffmann-Jørgensen, J. and Pisier, G. (1976). The law of large numbers and the central limit theorem in Banach spaces., Ann. Probab. 4 587–599.
  • [31] Horgan, D. and Murphy, C. C. (2013). On the Convergence of the Chi Square and Noncentral Chi Square Distributions to the Normal Distribution., IEEE Communications Letters 17 2233–2236.
  • [32] Horn, R. A. and Johnson, C. R. (1985)., Matrix analysis. Cambridge University Press, Cambridge.
  • [33] Jain, N. C. and Marcus, M. B. (1975). Integrability of infinite sums of independent vector-valued random variables., Trans. Amer. Math. Soc. 212 1–36.
  • [34] Ji, X. and Ruymgaart, F. H. (2008). Fréchet-differentiation of functions of operators with application to testing the equality of two covariance operators. In, Journal of Physics: Conference Series 124 012028. IOP Publishing.
  • [35] Jin, G. and Matthews, D. (2014). Reliability demonstration for long-life products based on degradation testing and a Wiener process model., IEEE Transactions on Reliability 63 781–797.
  • [36] John, S. (1971). Some optimal multivariate tests., Biometrika 58 123–127.
  • [37] Kato, T. (1995)., Perturbation theory for linear operators. Classics in Mathematics. Springer-Verlag, Berlin Reprint of the 1980 edition.
  • [38] Koroljuk, V. S. and Borovskich, Y. V. (1994)., Theory of $U$-statistics. Mathematics and its Applications 273. Kluwer Academic Publishers Group, Dordrecht. Translated from the 1989 Russian original by P. V. Malyshev and D. V. Malyshev and revised by the authors.
  • [39] Kosorok, M. R. (2008)., Introduction to empirical processes and semiparametric inference. Springer Series in Statistics. Springer, New York.
  • [40] Kotevski, Z. and Mitrevski, P. (2013). Hybrid fluid modeling approach for performance analysis of P2P live video streaming systems., Peer-to-Peer Networking and Applications 7 410–426.
  • [41] Küchler, U. and Tappe, S. (2013). Tempered stable distributions and processes., Stochastic Process. Appl. 123 4256–4293.
  • [42] Ledoit, O. and Wolf, M. (2002). Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size., Ann. Statist. 30 1081–1102.
  • [43] Li, K. (2014). Second-order asymptotics for quantum hypothesis testing., Ann. Statist. 42 171–189.
  • [44] Michel, R. (1981). On the constant in the nonuniform version of the Berry-Esseen theorem., Z. Wahrsch. Verw. Gebiete 55 109–117.
  • [45] MolavianJazi, E. (2015). Private, communication.
  • [46] Muirhead, R. J. (1982)., Aspects of multivariate statistical theory. John Wiley & Sons, Inc., New York Wiley Series in Probability and Mathematical Statistics.
  • [47] Nagao, H. (1973). On some test criteria for covariance matrix., Ann. Statist. 1 700–709.
  • [48] Noether, G. E. (1955). On a theorem of Pitman., Ann. Math. Statist. 26 64–68.
  • [49] Olkin, I. and Pukelsheim, F. (1982). The distance between two random vectors with given dispersion matrices., Linear Algebra Appl. 48 257–263.
  • [50] Petrov, V. V. (1975)., Sums of independent random variables. Springer-Verlag, New York. Translated from the Russian by A. A. Brown, Ergebnisse der Mathematik und ihrer Grenzgebiete, Band 82.
  • [51] Pfanzagl, J. (1971). The Berry-Esseen bound for minimum contrast estimates., Metrika 17 82–91.
  • [52] Pfanzagl, J. (1972/73). The accuracy of the normal approximation for estimates of vector parameters., Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 25 171–198.
  • [53] Pinelis, I. Comparing the asymptotic closeness of the distributions of Pearson’s and Fisher’s correlation statistics to normality, (draft).
  • [54] Pinelis, I. (2009). On the Bennett-Hoeffding inequality., http://arxiv.org/abs/0902.4058; a shorter version appeared in [62].
  • [55] Pinelis, I. (2011). Improved nonuniform Berry–Esseen-type bounds (preprint)., http://arxiv.org/abs/1109.0741.
  • [56] Pinelis, I. (2011). Exact lower bounds on the exponential moments of truncated random variables., J. Appl. Probab. 48 547–560.
  • [57] Pinelis, I. (2011). Exact bounds on the truncated-tilted mean, with applications (preprint)., http://arxiv.org/abs/1103.3683.
  • [58] Pinelis, I. (2012). An exact bound on the truncated-tilted mean for symmetric distributions (preprint)., http://arxiv.org/abs/1205.5234.
  • [59] Pinelis, I. (2013). An optimal bound on the quantiles of a certain kind of distributions (preprint)., http://arxiv.org/pdf/1301.0294.pdf.
  • [60] Pinelis, I. (2013). On the nonuniform Berry–Esseen bound., http://arxiv.org/abs/1301.2828.
  • [61] Pinelis, I. (2013). More on the nonuniform Berry–Esseen bound., http://arxiv.org/abs/1302.0516.
  • [62] Pinelis, I. (2014). On the Bennett-Hoeffding inequality., Ann. Inst. H. Poincaré Probab. Statist. 50 15–27.
  • [63] Pinelis, I. (2015). Exact Rosenthal-type bounds., Ann. Probab. 43 2511–2544.
  • [64] Pinelis, I. (2016). Optimal-order bounds on the rate of convergence to normality for maximum likelihood estimators., http://arxiv.org/abs/1601.02177.
  • [65] Pinelis, I. and Molzon, R. (2016). Optimal-order bounds on the rate of convergence to normality in the multivariate delta method (preprint)., http://arxiv.org/abs/0906.0177v5.
  • [66] Pinelis, I. F. (1986). Probability inequalities for sums of independent random variables with values in a Banach space., Math. Notes 39 241–244.
  • [67] Pinelis, I. F. (2015). Rosenthal-type inequalities for martingales in 2-smooth Banach spaces., Theory Probab. Appl. 59 699–706.
  • [68] Pinelis, I. F. and Sakhanenko, A. I. (1986). Remarks on inequalities for large deviation probabilities., Theory Probab. Appl. 30 143–148.
  • [69] Pinelis, I. F. and Utev, S. A. (1989). Exact exponential bounds for sums of independent random variables., Theory Probab. Appl. 34 340–346.
  • [70] Rippl, T., Munk, A. and Sturm, A. (2013). Limit laws of the empirical Wasserstein distance: Gaussian distributions (preprint)., http://arxiv.org/pdf/1507.04090v1.pdf.
  • [71] Römisch, W. (2006). Delta method, infinite dimensional., Encyclopedia of Statistical Sciences.
  • [72] Rosenthal, H. P. (1970). On the subspaces of $L^p$ $(p>2)$ spanned by sequences of independent random variables., Israel J. Math. 8 273–303.
  • [73] Shao, Q.-M., Wang, Q. et al. (2013). Self-normalized limit theorems: A survey., Probability Surveys 10 69–93.
  • [74] Shevtsova, I. (2011). On the absolute constants in the Berry-Esseen type inequalities for identically distributed summands (preprint)., http://arxiv.org/abs/1111.6554.
  • [75] Small, C. G. (2010)., Expansions and asymptotics for statistics. Monographs on Statistics and Applied Probability 115. CRC Press, Boca Raton, FL.
  • [76] Srivastava, M. S. (2005). Some tests concerning the covariance matrix in high dimensional data., J. Japan Statist. Soc. 35 251–272.
  • [77] Wasserman, L., Kolar, M. and Rinaldo, A. (2014). Berry-Esseen bounds for estimating undirected graphs., Electron. J. Stat. 8 1188–1224.
  • [78] Zalesskii, B. A. (1988). On the accuracy of normal approximation in Banach spaces., Theory Probab. Appl. 33 239–247.
  • [79] Zalesskii, B. A. (1990). The accuracy of Gaussian approximation in Banach spaces., Theory Probab. Appl. 34 747–748.
  • [80] Zeifman, A., Korolev, V., Satin, Y., Korotysheva, A. and Bening, V. (2014). Perturbation bounds and truncations for a class of Markovian queues., Queueing Syst. 76 205–221.