## Electronic Journal of Probability

### Coalescent processes in subdivided populations subject to recurrent mass extinctions

#### Abstract

We investigate the infinitely many demes limit of the genealogy of a sample of individuals from a subdivided population that experiences sporadic mass extinction events. By exploiting a separation of time scales that occurs within a class of structured population models generalizing Wright's island model, we show that as the number of demes tends to infinity, the limiting form of the genealogy can be described in terms of the alternation of instantaneous scattering phases that depend mainly on local demographic processes, and extended collecting phases that are dominated by global processes. When extinction and recolonization events are local, the genealogy is described by Kingman's coalescent, and the scattering phase influences only the overall rate of the process. In contrast, if the demes left vacant by a mass extinction event are recolonized by individuals emerging from a small number of demes, then the limiting genealogy is a coalescent process with simultaneous multiple mergers (a $\Xi$-coalescent). In this case, the details of the within-deme population dynamics influence not only the overall rate of the coalescent process, but also the statistics of the complex mergers that can occur within sample genealogies. These results suggest that the combined effects of geography and disturbance could play an important role in producing the unusual patterns of genetic variation documented in some marine organisms with high fecundity.

#### Article information

Source
Electron. J. Probab., Volume 14 (2009), paper no. 9, 242-288.

Dates
Accepted: 29 January 2009
First available in Project Euclid: 1 June 2016

https://projecteuclid.org/euclid.ejp/1464819470

Digital Object Identifier
doi:10.1214/EJP.v14-595

Mathematical Reviews number (MathSciNet)
MR2471665

Zentralblatt MATH identifier
1190.60066

Subjects
Primary: 60J25: Continuous-time Markov processes on general state spaces
Secondary: 60J75: Jump processes

Rights

#### Citation

Taylor, Jesse; Véber, Amandine. Coalescent processes in subdivided populations subject to recurrent mass extinctions. Electron. J. Probab. 14 (2009), paper no. 9, 242--288. doi:10.1214/EJP.v14-595. https://projecteuclid.org/euclid.ejp/1464819470

#### References

• C. Cannings. The Latent Roots of Certain Markov Chains Arising in Genetics: A New Approach, I. Haploid Models. Adv. Appl. Prob., 6:260–290, 1974. (49 #8689).
• J.T. Cox. Coalescing random walks and voter model consensus times on the torus in Zd. Ann. Probab., 17:1333–1366, 1989..
• J.T. Cox and R. Durrett. The stepping stone model: New formulas expose old myths. Ann. Appl. Probab., 12:1348–1377, 2002..
• B. Eldon and J. Wakeley. Coalescent Processes When the Distribution of Offspring Number Among Individuals is Highly Skewed. Genetics, 172:2621–2633, 2006.
• S.N. Ethier and T.G. Kurtz. Markov processes: characterization and convergence. Wiley, 1986.
• S.N. Ethier and T. Nagylaki. Diffusion approximation of Markov chains with two time scales and applications to population genetics. Adv. Appl. Probab., 12:14–49, 1980..
• R. Fisher. The Genetical Theory of Natural Selection. Clarenson, Oxford, 1930..
• A. Greven, V. Limic and A. Winter. Coalescent processes arising in a study of diffusive clustering. Preprint, 2007.
• J.F.C. Kingman. The coalescent. Stoch. Proc. Appl., 13:235–248, 1982..
• M. Möhle. Ancestral Processes in Population Genetics - the Coalescent. J. Theoret. Biol., 204:629–638, 2000.
• M. Möhle. On sampling distributions for coalescent processes with simultaneous multiple collisions. Bernoulli, 12:35–53, 2006.
• M. Möhle and S. Sagitov. A Classification of Coalescent Processes for Haploid Exchangeable Population Models. Ann. Probab., 29:1547–1562, 2001..
• M. Nordborg. Coalescent theory, Chapter 7 in Handbook of Statistical Genetics. Wiley, Chichester, UK, 2001.
• M. Nordborg and S.M. Krone. Separation of time scales and convergence to the coalescent in structured populations. In Modern Developments in Theoretical Population Genetics: The Legacy of Gustave Malécot. Oxford University Press, Oxford, 2002.
• M. Notohara. The coalescent and the genealogical process in geographically structured populations. J. Math. Biol., 29:59–75, 1990..
• J. Pitman. Coalescents with Multiple Collisions. Ann. Probab., 27:1870–1902, 1999..
• S. Sagitov. The general coalescent with asynchronous mergers of ancestral lines. J. Appl. Probab., 36:1116–1125, 1999..
• O. Sargsyan and J. Wakeley. A coalescent process with simultaneous multiple mergers for approximating the gene genealogies of many marine organisms. Theor. Popul. Biol., 74:104–114, 2008.
• J. Schweinsberg. Coalescents with simultaneous multiple collisions. Electr. J. Probab., 5:1–50, 2000..
• J. Schweinsberg. Coalescent processes obtained from supercritical Galton-Watson processes. Stochastic Process. Appl., 106:107–139, 2003..
• W.P. Sousa. The role of disturbance in natural communities. Annual Review of Ecology and Systematics, 15:353–391, 1984.
• A.W. van der Vaart. Asymptotic Statistics. Cambridge University Press, 1998..
• J. Wakeley. Segregating sites in Wright's island model. Theor. Popul. Biol., 53:166–175, 1998.
• J. Wakeley. Nonequilibrium Migration in Human History. Genetics, 153:1863–1871, 1999.
• J. Wakeley. Metapopulation models for historical inference. Mol. Ecol., 13:865–875, 2004.
• J. Wakeley and N. Aliacar. Gene Genealogies in a Metapopulation. Genetics, 159:893–905, 2001.
• H.M. Wilkinson-Herbots. Genealogy and subpopulation differentiation under various models of population structure. J. Math. Biol., 37:535–585, 1998.
• S. Wright. Evolution in Mendelian populations. Genetics, 16:97–159, 1931.
• I. Zähle, J.T. Cox and R. Durrett. The stepping stone model II: Genealogies and the infinite sites model. Ann. Appl. Probab., 15:671–699, 2005..