## Probability Surveys

- Probab. Surveys
- Volume 14 (2017), 1-52.

### Stein’s method for comparison of univariate distributions

Christophe Ley, Gesine Reinert, and Yvik Swan

**Full-text: Open access**

#### Abstract

We propose a new general version of Stein’s method for univariate distributions. In particular we propose a canonical definition of the *Stein operator* of a probability distribution which is based on a linear difference or differential-type operator. The resulting *Stein identity* highlights the unifying theme behind the literature on Stein’s method (both for continuous and discrete distributions). Viewing the Stein operator as an operator acting on pairs of functions, we provide an extensive toolkit for distributional comparisons. Several abstract approximation theorems are provided. Our approach is illustrated for comparison of several pairs of distributions: normal vs normal, sums of independent Rademacher vs normal, normal vs Student, and maximum of random variables vs exponential, Fréchet and Gumbel.

#### Article information

**Source**

Probab. Surveys, Volume 14 (2017), 1-52.

**Dates**

Received: April 2016

First available in Project Euclid: 9 January 2017

**Permanent link to this document**

https://projecteuclid.org/euclid.ps/1483952471

**Digital Object Identifier**

doi:10.1214/16-PS278

**Mathematical Reviews number (MathSciNet)**

MR3595350

**Zentralblatt MATH identifier**

06673431

**Subjects**

Primary: 60B10: Convergence of probability measures

Secondary: 60E15: Inequalities; stochastic orderings 60E05: Distributions: general theory 60F05: Central limit and other weak theorems

**Keywords**

Density approach Stein’s method comparison of distributions

#### Citation

Ley, Christophe; Reinert, Gesine; Swan, Yvik. Stein’s method for comparison of univariate distributions. Probab. Surveys 14 (2017), 1--52. doi:10.1214/16-PS278. https://projecteuclid.org/euclid.ps/1483952471

#### References

- [1] Afendras, G., Balakrishnan, N. and Papadatos, N. (2014). Orthogonal polynomials in the cumulative Ord family and its application to variance bounds, preprint arXiv:1408.1849.arXiv: 1408.1849
- [2] Afendras, G., Papadatos, N. and Papathanasiou, V. (2011). An extended Stein-type covariance identity for the Pearson family with applications to lower variance bounds,
*Bernoulli***17**, 507–529. - [3] Arras, B., Azmoodeh, E., Poly, G. and Swan, Y. (2016). Stein’s method on the second Wiener chaos: 2-Wasserstein distance, preprint arXiv:1601.03301.arXiv: 1601.03301
- [4] Baldi, P., Rinott, Y. and Stein, C. (1989). A normal approximation for the number of local maxima of a random function on a graph,
*Probability, Statistics, and Mathematics, Academic Press, Boston, MA*, 59–81. - [5] Barbour, A. D. (1990). Stein’s method for diffusion approximations,
*Probability Theory and Related Fields***84**, 297–322. - [6] Barbour, A. D. and Chen L. H. Y. (2005).
*An introduction to Stein’s method*,*Lecture Notes Series Inst. Math. Sci. Natl. Univ. Singap.***4**, Singapore University Press, Singapore. - [7] Barbour, A. D., Gan, H. L. and Xia, A. (2015). Stein factors for negative binomial approximation in Wasserstein distance,
*Bernoulli***21**, 1002–1013. - [8] Barbour, A. D., Holst, L. and Janson, S. (1992).
*Poisson approximation*, Oxford Studies in Probability,**2**, The Clarendon Press Oxford University Press, New York, Oxford Science Publications. - [9] Barbour, A. D. and Eagleson, G. K. (1985). Multiple comparisons and sums of dissociated random variables,
*Advances in Applied Probability***17**, 147–162. - [10] Barbour, A. D., Luczak M. J. and Xia A. (2015). Multivariate approximation in total variation, preprint arXiv:1512.07400.arXiv: 1512.07400
- [11] Bonis T. (2015). Stable measures and Stein’s method: rates in the Central Limit Theorem and diffusion approximation, preprint arXiv:1506.06966.arXiv: 1506.06966
- [12] Brown, T. C. and Xia, A. (1995). On Stein-Chen factors for Poisson approximation,
*Statistics & Probability Letters***23**, 327–332. - [13] Cacoullos, T., Papadatos, N. and Papathanasiou, V. (2001). An application of a density transform and the local limit theorem,
*Teor. Veroyatnost. i Primenen.***46**, 803–810. - [14] Cacoullos, T. and Papathanasiou, V. (1989). Characterizations of distributions by variance bounds,
*Statistics & Probability Letters***7**, 351–356.Mathematical Reviews (MathSciNet): MR1001133

Digital Object Identifier: doi:10.1016/0167-7152(89)90050-3 - [15] Cacoullos, T., Papadatos, N. and Papathanasiou, V. (1998). Variance inequalities for covariance kernels and applications to central limit theorems,
*Theory of Probability & Its Applications***42**, 149–155. - [16] Cacoullos, T. and Papathanasiou, V. (1995). A generalization of covariance identity and related characterizations,
*Mathematical Methods of Statistics***4**, 106–113. - [17] Cacoullos, T., Papathanasiou, V. and Utev, S. A. (1994). Variational inequalities with examples and an application to the central limit theorem,
*Annals of Probability***22**, 1607–1618. - [18] Chatterjee, S. (2014). A short survey of Stein’s method,
*Proceedings of ICM 2014*, to appear. - [19] Chatterjee, S., Fulman, J. and Röllin, A. (2011). Exponential approximation by exchangeable pairs and spectral graph theory,
*ALEA Latin American Journal of Probability and Mathematical Statistics***8**, 1–27. - [20] Chatterjee, S. and Meckes, E. (2008). Multivariate normal approximation using exchangeable pairs,
*ALEA Latin American Journal of Probability and Mathematical Statistics***4**, 257–283. - [21] Chatterjee, S. and Shao, Q.-M. (2011). Nonnormal approximation by Stein’s method of exchangeable pairs with application to the Curie-Weiss model,
*Annals of Applied Probability***21**, 464–483. - [22] Chen, L. H. Y. (1975). Poisson approximation for dependent trials,
*Annals of Probability***3**, 534–545. - [23] Chen, L. H. Y. (1980). An inequality for multivariate normal distribution,
*Technical report, MIT*. - [24] Chen, L. H. Y., Goldstein, L. and Shao, Q.-M. (2011).
*Normal approximation by Stein’s method*, Probability and its Applications (New York), Springer, Heidelberg. - [25] Daly, F. (2008). Upper bounds for Stein-type operators,
*Electronic Journal of Probability***13**, 566–587. - [26] Diaconis, P. and Zabell, S. (1991). Closed form summation for classical distributions: variations on a theme of de Moivre,
*Statistical Science***6**, 284–302. - [27] Döbler, C. (2012). On rates of convergence and Berry-Esseen bounds for random sums of centered random variables with finite third moments, preprint arXiv:1212.5401.arXiv: 1212.5401
- [28] Döbler, C. (2015). Stein’s method of exchangeable pairs for the beta distribution and generalizations,
*Electronic Journal of Probability***20**, 1–34. - [29] Döbler, C., Gaunt, R. E. and Vollmer, S. J. (2015). An iterative technique for bounding derivatives of solutions of Stein equations, preprint arXiv:1510.02623.arXiv: 1510.02623
- [30] Eden, R. and Viquez, J. (2015). Nourdin-Peccati analysis on Wiener and Wiener-Poisson space for general distributions,
*Stochastic Processes and their Applications***125**, 182–216. - [31] Ehm, W. (1991). Binomial approximation to the Poisson binomial distribution,
*Statistics & Probability Letters***11**, 7–16. - [32] Eichelsbacher, P. and Löwe, M. (2010). Stein’s method for dependent random variables occurring in statistical mechanics,
*Electronic Journal of Probability***15**, 962–988. - [33] Eichelsbacher, P. and Martschink, B. (2014). Rates of convergence in the Blume–Emery–Griffiths model,
*Journal of Statistical Physics***154**, 1483–1507. - [34] Eichelsbacher, P. and Reinert, G. (2008). Stein’s method for discrete Gibbs measures,
*Annals of Applied Probability***18**, 1588–1618.Mathematical Reviews (MathSciNet): MR2434182

Digital Object Identifier: doi:10.1214/07-AAP0498

Project Euclid: euclid.aoap/1216677133 - [35] Fulman, J. and Goldstein, L. (2015). Stein’s method and the rank distribution of random matrices over finite fields,
*Annals of Probability***43**, 1274–1314. - [36] Fulman, J. and Goldstein, L. (2014). Stein’s method, semicircle distribution, and reduced decompositions of the longest element in the symmetric group, preprint arXiv:1405.1088.arXiv: 1405.1088
- [37] Gaunt, R. E. (2016). On Stein’s method for products of normal random variables and zero bias couplings,
*Bernoulli*, to appear.Mathematical Reviews (MathSciNet): MR3654808

Digital Object Identifier: doi:10.3150/16-BEJ848

Project Euclid: euclid.bj/1495505094 - [38] Gaunt, R. E. (2014). Variance-Gamma approximation via Stein’s method,
*Electronic Journal of Probability***19**, 1–33. - [39] Gaunt, R. E., Mijoule, G. and Swan, Y. (2016). Stein operators for product distributions, with applications. preprint arXiv:1604.06819.arXiv: 1604.06819
- [40] Gibbs, A. L. and Su, F. E. (2002). On choosing and bounding probability metrics,
*International Statistical Review / Revue Internationale de Statistique***70**, 419–435. - [41] Goldstein, L. and Reinert, G. (1997). Stein’s method and the zero bias transformation with application to simple random sampling,
*Annals of Applied Probability***7**, 935–952. - [42] Goldstein, L. and Reinert, G. (2005). Distributional transformations, orthogonal polynomials, and Stein characterizations,
*Journal of Theoretical Probability***18**, 237–260. - [43] Goldstein, L. and Reinert, G. (2013). Stein’s method for the Beta distribution and the Pólya-Eggenberger urn,
*Journal of Applied Probability***50**, 1187–1205. - [44] Goldstein, L. and Rinott, Y. (1996). Multivariate normal approximations by Stein’s method and size bias couplings,
*Journal of Applied Probability***33**, 1–17. - [45] Gorham, J. and Mackey, L. (2015). Measuring sample quality with Stein’s method. In:
*Advances in Neural Information Processing Systems (NIPS)*, 226–234. - [46] Götze, F. and Tikhomirov, A. N. (2003). Rate of convergence to the semi-circular law,
*Probability Theory and Related Fields***127**, 228–276. - [47] Götze, F. and Tikhomirov, A. N. (2006). Limit theorems for spectra of random matrices with martingale structure,
*Teor. Veroyatnost. i Primenen.***51**, 171–192. - [48] Götze, F. (1991). On the rate of convergence in the multivariate CLT,
*Annals of Probability***19**, 724–739. - [49] Haagerup, U. and Thorbjørnsen, S. (2012). Asymptotic expansions for the Gaussian unitary ensemble,
*Infinite Dimensional Analysis, Quantum Probability and Related Topics***15**, no. 01. - [50] Hall, W. J. and Wellner, J. A. (1979). The rate of convergence in law of the maximum of an exponential sample,
*Statistica Neerlandica***33**, 151–154. - [51] Hillion, E., Johnson, O. and Yu, Y. (2014). A natural derivative on [0, n] and a binomial Poincaré inequality.
*ESAIM: Probability and Statistics***18**, 703–712. - [52] Holmes, S. (2004). Stein’s method for birth and death chains,
*Stein’s Method: Expository Lectures and Applications, IMS Lecture Notes Monogr. Ser.,***46**, 45–67. - [53] Johnson, O. (2004).
*Information Theory and the Central Limit Theorem*, Imperial College Press, London. - [54] Johnson, O. and Barron, A. (2004). Fisher information inequalities and the central limit theorem,
*Probability Theory and Related Fields***129**, 391–409.Mathematical Reviews (MathSciNet): MR2128239

Digital Object Identifier: doi:10.1007/s00440-004-0344-0 - [55] Johnson, R. W. (1993). A note on variance bounds for a function of a Pearson variate,
*Statistics & Risk Modeling***11**, 273–278. - [56] Klaassen, C. A. J. (1985). On an inequality of Chernoff,
*Annals of Probability***13**, 966–974. - [57] Korwar, R. M. (1991). On characterizations of distributions by mean absolute deviation and variance bounds,
*Annals of the Institute of Statistical Mathematics***43**, 287–295. - [58] Kusuoka, S. and Tudor, C. A. (2012). Stein’s method for invariant measures of diffusions via Malliavin calculus,
*Stochastic Processes and Their Applications***122**, 1627–1651. - [59] Kusuoka, S. and Tudor, C. A. (2013). Extension of the fourth moment theorem to invariant measures of diffusions, preprint arXiv:1310.3785.arXiv: 1310.3785
- [60] Ledoux, M., Nourdin, I. and Peccati, G. (2015). Stein’s method, logarithmic Sobolev and transport inequalities,
*Geometric and Functional Analysis***25**, 256–306. - [61] Lefèvre, C., Papathanasiou, V. and Utev, S. (2002). Generalized Pearson distributions and related characterization problems,
*Annals of the Institute of Statistical Mathematics***54**, 731–742. - [62] Ley, C., Reinert, G. and Swan, Y. (2016). Distances between nested densities and a measure of the impact of the prior in Bayesian statistics,
*Annals of Applied Probability*, to appear. - [63] Ley, C. and Swan, Y. (2016). A general parametric Stein characterization,
*Statistics & Probability Letters***111**, 67–71.Mathematical Reviews (MathSciNet): MR3474784

Digital Object Identifier: doi:10.1016/j.spl.2016.01.008 - [64] Ley, C. and Swan, Y. (2013). Local Pinsker inequalities via Stein’s discrete density approach,
*IEEE Transactions on Information Theory***59**, 5584–4491. - [65] Ley, C. and Swan, Y. (2013). Stein’s density approach and information inequalities,
*Electronic Communications in Probability***18**, 1–14. - [66] Ley, C. and Swan, Y. (2016). Parametric Stein operators and variance bounds,
*Brazilian Journal of Probability and Statistics***30**, 171–195. - [67] Loh, W.-L. (2004). On the characteristic function of Pearson type IV distributions,
*A Festschrift for Herman Rubin, Institute of Mathematical Statistics*, 171–179. - [68] Luk, H. M. (1994).
*Stein’s method for the Gamma distribution and related statistical applications*, Ph.D. thesis, University of Southern California. - [69] Mackey, L. and Gorham, J. (2016). Multivariate Stein factors for a class of strongly log-concave distributions. Electronic Communications in Probability
**21**paper no. 56, 14 pp. - [70] Nourdin, I. and Peccati, G. (2009). Stein’s method on Wiener chaos,
*Probability Theory and Related Fields***145**, 75–118.Mathematical Reviews (MathSciNet): MR2520122

Digital Object Identifier: doi:10.1007/s00440-008-0162-x - [71] Nourdin, I. and Peccati, G. (2012).
*Normal approximations with Malliavin calculus: from Stein’s method to universality*, Cambridge Tracts in Mathematics, Cambridge University Press. - [72] Nourdin, I., Peccati, G. and Reinert, G. (2009). Second order Poincaré inequalities and CLTs on Wiener space,
*Journal of Functional Analysis***257**, 593–609. - [73] Nourdin, I., Peccati, G. and Swan, Y. (2014). Entropy and the fourth moment phenomenon,
*Journal of Functional Analysis***266**, 3170–3207. - [74] Nourdin, I., Peccati, G. and Swan, Y. (2014). Integration by parts and representation of information functionals,
*IEEE International Symposium on Information Theory (ISIT)*, 2217–2221. - [75] Novak, S. Y. (2011).
*Extreme Value Methods with Applications to Finance*, Chapman & Hall/CRC Press, Boca Raton. - [76] Ord, J. K. (1967). On a system of discrete distributions,
*Biometrika***54**, 649–656.Mathematical Reviews (MathSciNet): MR224192 - [77] Papadatos, N. and Papathanasiou, V. (1995). Distance in variation between two arbitrary distributions via the associated w-functions,
*Theory of Probability & Its Applications***40**, 567–575. - [78] Papathanasiou, V. (1995). A characterization of the Pearson system of distributions and the associated orthogonal polynomials,
*Annals of the Institute of Statistical Mathematics***47**, 171–176. - [79] Peköz, E. and Röllin, A. (2011). New rates for exponential approximation and the theorems of Rényi and Yaglom,
*Annals of Probability***39**, 587–608.Mathematical Reviews (MathSciNet): MR2789507

Digital Object Identifier: doi:10.1214/10-AOP559

Project Euclid: euclid.aop/1298669174 - [80] Peköz, E. Röllin, A. and Ross, N. (2013). Degree asymptotics with rates for preferential attachment random graphs,
*Annals of Applied Probability***23**, 1188–1218. - [81] Pickett, A. (2004).
*Rates of convergence of $\chi^{2}$ approximations via Stein’s method*, Ph.D. thesis, Lincoln College, University of Oxford. - [82] Pike, J. and Ren, H. (2014). Stein’s method and the Laplace distribution,
*ALEA Latin American Journal of Probability and Mathematical Statistics***11**, 571–587. - [83] Rachev, S. T. (1991).
*Probability Metrics and the Stability of Stochastic Models*, vol. 334, Wiley New York. - [84] Reinert, G. (1998). Couplings for normal approximations with Stein’s method,
*DIMACS Ser. Discrete Math. Theoret. Comput. Sci.*,**41**, 193–207. - [85] Reinert, G. and Röllin, A. (2009). Multivariate normal approximation with Stein’s method of exchangeable pairs under a general linearity condition,
*Annals of Probability***37**, 2150–2173. - [86] Röllin, A. (2012). On magic factors and the construction of examples with sharp rates in Stein’s method.
*Probability Approximations and Beyond, Lecture Notes in Statistics***205**, Springer. - [87] Ross, N. (2011). Fundamentals of Stein’s method,
*Probability Surveys***8**, 210–293. - [88] Schoutens, W. (2001). Orthogonal polynomials in Stein’s method,
*Journal of Mathematical Analysis and Applications***253**, 515–531. - [89] Shimizu, R. (1975). On Fisher’s amount of information for location family,
*A Modern Course on Statistical Distributions in Scientific Work*, Springer, 305–312. - [90] Stein, C. (1972). A bound for the error in the normal approximation to the distribution of a sum of dependent random variables,
*Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability (Univ. California, Berkeley, Calif., 1970/1971), Vol. II: Probability theory (Berkeley, Calif.), Univ. California Press*, 583–602. - [91] Stein, C. (1986).
*Approximate computation of expectations*, Institute of Mathematical Statistics Lecture Notes—Monograph Series, 7, Institute of Mathematical Statistics, Hayward, CA. - [92] Stein, C., Diaconis, P., Holmes, S. and Reinert, G. (2004). Use of exchangeable pairs in the analysis of simulations,
*Stein’s method: expository lectures and applications (Persi Diaconis and Susan Holmes, eds.), IMS Lecture Notes Monogr. Ser, vol. 46, Beachwood, Ohio, USA: Institute of Mathematical Statistics*, 1–26. - [93] Upadhye, N. S., Cekanavicius, V. and Vellaisamy, P. (2016). On Stein operators for discrete approximations,
*Bernoulli*, to appear.

#### The Institute of Mathematical Statistics and the Bernoulli Society

### More like this

- Stein's Method and Stochastic Analysis of Rademacher Functionals

Reinert, Gesine, Nourdin, Ivan, and Peccati, Giovanni, Electronic Journal of Probability, 2010 - Fundamentals of Stein’s method

Ross, Nathan, Probability Surveys, 2011 - Multivariate normal approximation with Stein’s method of exchangeable pairs under a general linearity condition

Reinert, Gesine and Röllin, Adrian, The Annals of Probability, 2009

- Stein's Method and Stochastic Analysis of Rademacher Functionals

Reinert, Gesine, Nourdin, Ivan, and Peccati, Giovanni, Electronic Journal of Probability, 2010 - Fundamentals of Stein’s method

Ross, Nathan, Probability Surveys, 2011 - Multivariate normal approximation with Stein’s method of exchangeable pairs under a general linearity condition

Reinert, Gesine and Röllin, Adrian, The Annals of Probability, 2009 - Stein's density approach and information inequalities

Ley, Christophe and Swan, Yvik, Electronic Communications in Probability, 2013 - Distances between nested densities and a measure of the impact of the prior in Bayesian statistics

Ley, Christophe, Reinert, Gesine, and Swan, Yvik, The Annals of Applied Probability, 2017 - Stein's method of exchangeable pairs for the Beta distribution and generalizations

Döbler, Christian, Electronic Journal of Probability, 2015 - New Nonparametric Tests of Multivariate Locations and Scales Using Data Depth

Li, Jun and Liu, Regina Y., Statistical Science, 2004 - Models for Paired Comparison Data: A Review with Emphasis on Dependent Data

Cattelan, Manuela, Statistical Science, 2012 - Stein’s method for birth and death chains

Holmes, Susan, Stein's Method, 2004 - Direct use of regression quantiles to construct confidence sets in linear models

Zhou, Kenneth Q. and Portnoy, Stephen L., The Annals of Statistics, 1996