The Annals of Probability

A general method for lower bounds on fluctuations of random variables

Sourav Chatterjee

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

There are many ways of establishing upper bounds on fluctuations of random variables, but there is no systematic approach for lower bounds. As a result, lower bounds are unknown in many important problems. This paper introduces a general method for lower bounds on fluctuations. The method is used to obtain new results for the stochastic traveling salesman problem, the stochastic minimal matching problem, the random assignment problem, the Sherrington–Kirkpatrick model of spin glasses, first-passage percolation and random matrices. A long list of open problems is provided at the end.

Article information

Source
Ann. Probab., Volume 47, Number 4 (2019), 2140-2171.

Dates
Received: August 2017
Revised: July 2018
First available in Project Euclid: 4 July 2019

Permanent link to this document
https://projecteuclid.org/euclid.aop/1562205705

Digital Object Identifier
doi:10.1214/18-AOP1304

Mathematical Reviews number (MathSciNet)
MR3980917

Subjects
Primary: 60E15: Inequalities; stochastic orderings 60C05: Combinatorial probability 60K35: Interacting random processes; statistical mechanics type models; percolation theory [See also 82B43, 82C43] 60B20: Random matrices (probabilistic aspects; for algebraic aspects see 15B52)

Keywords
Variance lower bound first-passage percolation random assignment problem stochastic minimal matching problem stochastic traveling salesman problem spin glass Sherrington–Kirkpatrick model random matrix determinant

Citation

Chatterjee, Sourav. A general method for lower bounds on fluctuations of random variables. Ann. Probab. 47 (2019), no. 4, 2140--2171. doi:10.1214/18-AOP1304. https://projecteuclid.org/euclid.aop/1562205705


Export citation

References

  • [1] Aizenman, M., Lebowitz, J. L. and Ruelle, D. (1987). Some rigorous results on the Sherrington–Kirkpatrick spin glass model. Comm. Math. Phys. 112 3–20.
  • [2] Aldous, D. (1992). Asymptotics in the random assignment problem. Probab. Theory Related Fields 93 507–534.
  • [3] Aldous, D. J. (2001). The $\zeta(2)$ limit in the random assignment problem. Random Structures Algorithms 18 381–418.
  • [4] Alexander, K. S. (1993). A note on some rates of convergence in first-passage percolation. Ann. Appl. Probab. 3 81–90.
  • [5] Alexander, K. S. (1997). Approximation of subadditive functions and convergence rates in limiting-shape results. Ann. Probab. 25 30–55.
  • [6] Alm, S. E. and Sorkin, G. B. (2002). Exact expectations and distributions for the random assignment problem. Combin. Probab. Comput. 11 217–248.
  • [7] Anderson, T. W. (2003). An Introduction to Multivariate Statistical Analysis, 3rd ed. Wiley-Interscience, Hoboken, NJ.
  • [8] Auffinger, A., Damron, M. and Hanson, J. (2015). Rate of convergence of the mean for sub-additive ergodic sequences. Adv. Math. 285 138–181.
  • [9] Auffinger, A., Damron, M. and Hanson, J. (2017). 50 Years of First-Passage Percolation. University Lecture Series 68. Amer. Math. Soc., Providence, RI.
  • [10] Benaï m, M. and Rossignol, R. (2008). Exponential concentration for first passage percolation through modified Poincaré inequalities. Ann. Inst. Henri Poincaré Probab. Stat. 44 544–573.
  • [11] Benjamini, I., Kalai, G. and Schramm, O. (2003). First passage percolation has sublinear distance variance. Ann. Probab. 31 1970–1978.
  • [12] Bollobás, B. and Janson, S. (1997). On the length of the longest increasing subsequence in a random permutation. In Combinatorics, Geometry and Probability (Cambridge, 1993) 121–128. Cambridge Univ. Press, Cambridge.
  • [13] Bose, A., Subhra Hazra, R. and Saha, K. (2010). Patterned random matrices and method of moments. In Proceedings of the International Congress of Mathematicians. Volume IV 2203–2231. Hindustan Book Agency, New Delhi.
  • [14] Boucheron, S., Lugosi, G. and Massart, P. (2013). Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford Univ. Press, Oxford.
  • [15] Cai, T. T., Liang, T. and Zhou, H. H. (2015). Law of log determinant of sample covariance matrix and optimal estimation of differential entropy for high-dimensional Gaussian distributions. J. Multivariate Anal. 137 161–172.
  • [16] Chatterjee, S. (2009). Disorder chaos and multiple valleys in spin glasses Preprint. Available at https://arxiv.org/abs/0907.3381.
  • [17] Chatterjee, S. (2013). The universal relation between scaling exponents in first-passage percolation. Ann. of Math. (2) 177 663–697.
  • [18] Chatterjee, S. (2014). Superconcentration and Related Topics. Springer, Cham.
  • [19] Chen, L. H. Y., Goldstein, L. and Shao, Q.-M. (2011). Normal Approximation by Stein’s Method. Springer, Heidelberg.
  • [20] Chen, W.-K., Dey, P. and Panchenko, D. (2017). Fluctuations of the free energy in the mixed $p$-spin models with external field. Probab. Theory Related Fields 168 41–53.
  • [21] Chen, W.-K., Handschy, M. and Lerman, G. (2018). On the energy landscape of the mixed even $p$-spin model. Probab. Theory Related Fields 171 53–95.
  • [22] Cox, J. T. and Durrett, R. (1981). Some limit theorems for percolation processes with necessary and sufficient conditions. Ann. Probab. 9 583–603.
  • [23] Damron, M., Hanson, J. and Sosoe, P. (2015). Sublinear variance in first-passage percolation for general distributions. Probab. Theory Related Fields 163 223–258.
  • [24] Gong, R., Houdré, C. and Lember, J. (2018). Lower bounds on the generalized central moments of the optimal alignments score of random sequences. J. Theoret. Probab. 31 643–683.
  • [25] Goodman, N. R. (1963). The distribution of the determinant of a complex Wishart distributed matrix. Ann. Math. Stat. 34 178–180.
  • [26] Hessler, M. (2009). Optimization, matroids and error-correcting codes. Ph.D. thesis, Linköping Univ.
  • [27] Hessler, M. and Wästlund, J. (2008). Concentration of the cost of a random matching problem. Preprint. Available at http://www.math.chalmers.se/~wastlund/martingale.pdf.
  • [28] Houdré, C. and Işlak, U. (2014). A central limit theorem for the length of the longest common subsequences in random words. Preprint. Available at https://arxiv.org/abs/1408.1559.
  • [29] Houdré, C. and Ma, J. (2016b). On the order of the central moments of the length of the longest common subsequences in random words. In High Dimensional Probability VII. Progress in Probability 71 105–136. Springer, Cham.
  • [30] Houdré, C. and Matzinger, H. (2016). On the variance of the optimal alignments score for binary random words and an asymmetric scoring function. J. Stat. Phys. 164 693–734.
  • [31] Janson, S. (1994). Self-couplings and the concentration function. Acta Appl. Math. 34 5–6.
  • [32] Janson, S. and Warnke, L. (2016). The lower tail: Poisson approximation revisited. Random Structures Algorithms 48 219–246.
  • [33] Le Cam, L. and Yang, G. L. (2000). Asymptotics in Statistics: Some Basic Concepts, 2nd ed. Springer, New York.
  • [34] Ledoux, M. (2001). The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89. Amer. Math. Soc., Providence, RI.
  • [35] Lember, J. and Matzinger, H. (2009). Standard deviation of the longest common subsequence. Ann. Probab. 37 1192–1235.
  • [36] Levin, D. A., Peres, Y. and Wilmer, E. L. (2009). Markov Chains and Mixing Times. Amer. Math. Soc., Providence, RI.
  • [37] Lévy, P. (1937). Théorie de L’addition des Variables Aléatoires. Gauthier-Villars, Paris.
  • [38] Linusson, S. and Wästlund, J. (2004). A proof of Parisi’s conjecture on the random assignment problem. Probab. Theory Related Fields 128 419–440.
  • [39] McBryan, O. A. and Spencer, T. (1977). On the decay of correlations in $\operatorname{SO}(n)$-symmetric ferromagnets. Comm. Math. Phys. 53 299–302.
  • [40] Mermin, N. D. (1967). Absence of ordering in certain classical systems. J. Math. Phys. 8 1061–1064.
  • [41] Mermin, N. D. and Wagner, H. (1966). Absence of ferromagnetism or antiferromagnetism in one- or two-dimensional isotropic Heisenberg models. Phys. Rev. Lett. 17 1133–1136.
  • [42] Mézard, M. and Parisi, G. (1985). Replicas and optimization. J. Phys. Lett. 46 771–778.
  • [43] Mézard, M. and Parisi, G. (1987). On the solution of the random link matching problem. J. Physique 48 1451–1459.
  • [44] Nair, C., Prabhakar, B. and Sharma, M. (2005). Proofs of the Parisi and Coppersmith–Sorkin random assignment conjectures. Random Structures Algorithms 27 413–444.
  • [45] Nakajima, S. (2017). Divergence of shape fluctuation in first passage percolation. Preprint. Available at https://arxiv.org/abs/1706.03493.
  • [46] Newman, C. M. and Piza, M. S. T. (1995). Divergence of shape fluctuations in two dimensions. Ann. Probab. 23 977–1005.
  • [47] Nguyen, H. H. and Vu, V. (2014). Random matrices: Law of the determinant. Ann. Probab. 42 146–167.
  • [48] Palassini, M. (2008). Ground-state energy fluctuations in the Sherrington–Kirkpatrick model. J. Stat. Mech. 2008 P10005.
  • [49] Panchenko, D. (2013). The Sherrington–Kirkpatrick Model. Springer, New York.
  • [50] Pastur, L. and Shcherbina, M. (2011). Eigenvalue Distribution of Large Random Matrices. Mathematical Surveys and Monographs 171. Amer. Math. Soc., Providence, RI.
  • [51] Peled, R. and Spinka, Y. Lectures on the Spin and Loop $O(n)$ Models. Preprint. Available at https://arxiv.org/abs/1708.00058.
  • [52] Pemantle, R. and Peres, Y. (1994). Planar first-passage percolation times are not tight. In Probability and Phase Transition (Cambridge, 1993). NATO Adv. Sci. Inst. Ser. C Math. Phys. Sci. 420 261–264. Kluwer Academic, Dordrecht.
  • [53] Petrov, V. V. (1975). Sums of Independent Random Variables. Ergebnisse der Mathematik und Ihrer Grenzgebiete 82. Springer, New York-Heidelberg. Translated from the Russian by A. A. Brown.
  • [54] Pfister, C. E. (1981). On the symmetry of the Gibbs states in two-dimensional lattice systems. Comm. Math. Phys. 79 181–188.
  • [55] Rhee, W. T. (1991). On the fluctuations of the stochastic traveling salesperson problem. Math. Oper. Res. 16 482–489.
  • [56] Sherrington, D. and Kirkpatrick, S. (1975). Solvable model of a spin glass. Phys. Rev. Lett. 35 1792–1796.
  • [57] Steele, J. M. (1997). Probability Theory and Combinatorial Optimization. CBMS-NSF Regional Conference Series in Applied Mathematics 69. SIAM, Philadelphia, PA.
  • [58] Talagrand, M. (1995). Concentration of measure and isoperimetric inequalities in product spaces. Publ. Math. Inst. Hautes Études Sci. 81 73–205.
  • [59] Talagrand, M. (2003). Spin Glasses: A Challenge for Mathematicians: Cavity and Mean Field Models. Ergebnisse der Mathematik und Ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd Series. A Series of Modern Surveys in Mathematics] 46. Springer, Berlin.
  • [60] Talagrand, M. (2011a). Mean Field Models for Spin Glasses. Vol. I. Basic Examples. Ergebnisse der Mathematik und Ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd Series. A Series of Modern Surveys in Mathematics] 54. Springer, Berlin.
  • [61] Talagrand, M. (2011b). Mean Field Models for Spin Glasses. Vol. II. Advanced Replica-Symmetry and Low Temperature. Ergebnisse der Mathematik und Ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd Series. A Series of Modern Surveys in Mathematics] 55. Springer, Heidelberg.
  • [62] Tao, T. and Vu, V. (2012). A central limit theorem for the determinant of a Wigner matrix. Adv. Math. 231 74–101.
  • [63] Wästlund, J. (2005). The variance and higher moments in the random assignment problem. Linköping Studies in Mathematics, no. 8.
  • [64] Wästlund, J. (2010). The mean field traveling salesman and related problems. Acta Math. 204 91–150.
  • [65] Wästlund, J. (2012). Replica symmetry of the minimum matching. Ann. of Math. (2) 175 1061–1091.
  • [66] Wehr, J. and Aizenman, M. (1990). Fluctuations of extensive functions of quenched random couplings. J. Stat. Phys. 60 287–306.
  • [67] Zhang, Y. (2006). The divergence of fluctuations for shape in first passage percolation. Probab. Theory Related Fields 136 298–320.