## Electronic Journal of Probability

### The Common Ancestor Process for a Wright-Fisher Diffusion

Jesse Taylor

#### Abstract

Rates of molecular evolution along phylogenetic trees are influenced by mutation, selection and genetic drift. Provided that the branches of the tree correspond to lineages belonging to genetically isolated populations (e.g., multi-species phylogenies), the interplay between these three processes can be described by analyzing the process of substitutions to the common ancestor of each population. We characterize this process for a class of diffusion models from population genetics theory using the structured coalescent process introduced by Kaplan et al. (1988) and formalized in Barton et al. (2004). For two-allele models, this approach allows both the stationary distribution of the type of the common ancestor and the generator of the common ancestor process to be determined by solving a one-dimensional boundary value problem. In the case of a Wright-Fisher diffusion with genic selection, this solution can be found in closed form, and we show that our results complement those obtained by Fearnhead (2002) using the ancestral selection graph. We also observe that approximations which neglect recurrent mutation can significantly underestimate the exact substitution rates when selection is strong. Furthermore, although we are unable to find closed-form expressions for models with frequency-dependent selection, we can still solve the corresponding boundary value problem numerically and then use this solution to calculate the substitution rates to the common ancestor. We illustrate this approach by studying the effect of dominance on the common ancestor process in a diploid population. Finally, we show that the theory can be formally extended to diffusion models with more than two genetic backgrounds, but that it leads to systems of singular partial differential equations which we have been unable to solve.

#### Article information

Source
Electron. J. Probab., Volume 12 (2007), paper no. 28, 808-847.

Dates
Accepted: 1 June 2007
First available in Project Euclid: 1 June 2016

https://projecteuclid.org/euclid.ejp/1464818499

Digital Object Identifier
doi:10.1214/EJP.v12-418

Mathematical Reviews number (MathSciNet)
MR2318411

Zentralblatt MATH identifier
1127.60079

Rights

#### Citation

Taylor, Jesse. The Common Ancestor Process for a Wright-Fisher Diffusion. Electron. J. Probab. 12 (2007), paper no. 28, 808--847. doi:10.1214/EJP.v12-418. https://projecteuclid.org/euclid.ejp/1464818499

#### References

• H. Akashi. Inferring Weak Selection from Patterns of Polymorphism and Divergence at Silent Sites in Drosophila DNA. Genetics, 139:1067–1076, 1995.
• Baake, Ellen; Georgii, Hans-Otto. Mutation, selection, and ancestry in branching models: a variational approach. J. Math. Biol. 54 (2007), no. 2, 257–303.
• N.H. Barton and A.M. Etheridge. The Effect of Selection on Genealogies. Genetics, 166:1115–1131, 2004.
• Barton, N. H.; Etheridge, A. M.; Sturm, A. K. Coalescence in a random background. Ann. Appl. Probab. 14 (2004), no. 2, 754–785.
• N.H. Barton and S.P. Otto. Evolution of Recombination Due to Random Drift. Genetics, 169:2353–2370, 2005.
• Birkhoff, Garrett; Rota, Gian-Carlo. Ordinary differential equations. Fourth edition. John Wiley & Sons, Inc., New York, 1989. xii+399 pp. ISBN: 0-471-86003-4
• C.D. Bustamante, J. Wakeley, S.Sawyer, and D.L. Hartl. Directional Selection and the Site-Frequency Spectrum. Genetics, 159:1779–1788, 2001.
• J.L. Cherry and J. Wakeley. A Diffusion Approximation for Selection and Drift in a Subdivided Population. Genetics, 163:421–428, 2003.
• G. Coop and R.C. Griffiths. Ancestral inference on gene trees under selection. Theor. Pop. Biol., 66:219–232, 2004.
• Donnelly, Peter; Kurtz, Thomas G. A countable representation of the Fleming-Viot measure-valued diffusion. Ann. Probab. 24 (1996), no. 2, 698–742.
• Donnelly, Peter; Kurtz, Thomas G. Genealogical processes for Fleming-Viot models with selection and recombination. Ann. Appl. Probab. 9 (1999), no. 4, 1091–1148.
• A.M. Etheridge. Evolution in Fluctuating Populations. In A.Bovier, F.Dunlop, F.den Hollander, A.van Enter, and J.Dalibard, editors, Mathematical Statistical Physics, volume 83, pages 489–545. Les Houches, Elsevier, 2005.
• Ethier, Stewart N.; Kurtz, Thomas G. Markov processes. Characterization and convergence. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley & Sons, Inc., New York, 1986. x+534 pp. ISBN: 0-471-08186-8
• Ewens, Warren J. Mathematical population genetics. I. Theoretical introduction. Second edition. Interdisciplinary Applied Mathematics, 27. Springer-Verlag, New York, 2004. xx+417 pp. ISBN: 0-387-20191-2
• Fearnhead, Paul. The common ancestor at a nonneutral locus. J. Appl. Probab. 39 (2002), no. 1, 38–54.
• S. Gavrilets. Perspective: Models of speciation: What have we learned in 40 years? Evolution, 57:2197–2215, 2003.
• Georgii, Hans-Otto; Baake, Ellen. Supercritical multitype branching processes: the ancestral types of typical individuals. Adv. in Appl. Probab. 35 (2003), no. 4, 1090–1110.
• J. H. Gillespie. The Causes of Molecular Evolution. Oxford University Press, Oxford, 1991.
• J. H. Gillespie. Population Genetics: A Concise Guide. Johns Hopkins University Press, Baltimore, 2004.
• Griffiths, Robert C.; Marjoram, Paul. An ancestral recombination graph. Progress in population genetics and human evolution (Minneapolis, MN, 1994), 257–270, IMA Vol. Math. Appl., 87, Springer, New York, 1997.
• Jagers, Peter. General branching processes as Markov fields. Stochastic Process. Appl. 32 (1989), no. 2, 183–212.
• Jagers, Peter. Stabilities and instabilities in population dynamics. J. Appl. Probab. 29 (1992), no. 4, 770–780.
• N. L. Kaplan, T. Darden, and R. R. Hudson. The coalescent process in models with selection. Genetics, 120:819–829, 1988.
• Karatzas, Ioannis; Shreve, Steven E. Brownian motion and stochastic calculus. Second edition. Graduate Texts in Mathematics, 113. Springer-Verlag, New York, 1991. xxiv+470 pp. ISBN: 0-387-97655-8
• Kimura, Motoo. Diffusion models in population genetics. J. Appl. Probability 1 1964 177–232.
• F. A. Kondrashov and E. V. Koonin. A common framework for understanding the origin of genetic dominance and evolutionary fates of gene duplications. Trends Genet., 20:287–291, 2004.
• R. D. Koyous, C. L. Althaus, and S. Bonhoeffer. Stochastic or deterministic: what is the effective population size of HIV-1? Trends Microbiol., 14:507–511, 2006.
• S. M. Krone and C. Neuhauser. Ancestral processes with selection. Theor. Pop. Biol., 51:210–237, 1997.
• R. Lande. Natural Selection and Random Genetic Drift in Phenotypic Evolution. Evolution, 30:314–334, 1976.
• Li, Zenghu; Shiga, Tokuzo; Yao, Lihua. A reversibility problem for Fleming-Viot processes. Electron. Comm. Probab. 4 (1999), 65–76 (electronic).
• M. Lynch and J. S. Conery. The Origins of Genome Complexity. Science, 302:1401–1404, 2003.
• G. McVean and J. Vieira. Inferring parameters of mutation, selection and demography from patterns of synonymous site evolution in Drosophila. Genetics, 157:245–257, 2001.
• Nelson, Edward. The adjoint Markoff process. Duke Math. J. 25 1958 671–690.
• C. Neuhauser. The ancestral graph and gene genealogy under frequency-dependent selection. Theor. Pop. Biol., 56:203–214, 1999.
• R. Nielsen and Z. Yang. Likelihood models for detecting positively selected amino acid sites and applications to the HIV-1 envelope gene. Genetics, 148:929–936, 1998.
• Norman, M. Frank. Ergodicity of diffusion and temporal uniformity of diffusion approximation. J. Appl. Probability 14 (1977), no. 2, 399–404.
• R. B. O'Hara. Comparing the effects of genetic drift and fluctuating selection on genotype frequency changes in the scarlet tiger moth. Proc. Roy. Soc. Lond. B, 272:211–217, 2005.
• S. P. Otto and M. C. Whitlock. The Probability of Fixation in Populations of Changing Size. Genetics, 146:723–733, 1997.
• N. Patterson, D. J. Richter, S. Gnerre, E. S. Lander, and D. Reich. Genetic evidence for complex speciation of humans and chimpanzees. Nature, 441:1103–1108, 2006.
• A. Poon and L. Chao. Drift Increases the Advantage of Sex in RNA Bacteriophage $\Phi 6$. Genetics, 166, 2004.
• W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. Numerical Recipes in C. The Art of Scientific Computing. Cambridge University Press, Cambridge, 1992.
• G. R. Price. Selection and Covariance. Nature, 227:520–521, 1970.
• A. Richman. Evolution of balanced genetic polymorphism. Mol. Ecol., 9:1953–963, 2000.
• D. Roze and F. Rousset. Selection and Drift in Subdidived Populations: A Straightforward Method for Deriving Diffusion Approximations and Applications Involving Dominance, Selfing and Local Extinctions. Genetics, 165:2153–2166, 2004.
• S. A. Sawyer and D. L. Hartl. Population Genetics of Polymorphism and Divergence. Genetics, 132:1161–1176, 1992.
• Shiga, Tokuzo. Diffusion processes in population genetics. J. Math. Kyoto Univ. 21 (1981), no. 1, 133–151.
• Stephens, Matthew; Donnelly, Peter. Ancestral inference in population genetics models with selection (with discussion). Aust. N. Z. J. Stat. 45 (2003), no. 4, 395–430.
• M. C. Whitlock. Fixation of New Alleles and the Extinction of Small Populations: Drift Load, Beneficial Alleles, and Sexual Selection. Evol., 54:1855–1861, 2000.
• S. Williamson, A. Fledel-Alon, and C. D. Bustamante. Population Genetics of Polymorphism and Divergence for Diploid Selection Models With Arbitrary Dominance. Genetics, 168:463–475, 2004.
• Z. Yang. Among-site rate variation and its impact on phylogenetic analyses. Trends Ecol. Evol., 11:367–372, 1996.
• A. Zharkikh. Estimation of Evolutionary Distances Between Nucleotide Sequences. J. Mol. Evol., 39:315–329, 1994.