Occupation laws for some time-nonhomogeneous Markov chains

Abstract

We consider finite-state time-nonhomogeneous Markov chains whose transition matrix at time $n$ is $I+G/n^z$ where $G$ is a ``generator'' matrix, that is $G(i,j)>0$ for $i,j$ distinct, and $G(i,i)= -\sum_{k\ne i} G(i,k)$, and $z>0$ is a strength parameter. In these chains, as time grows, the positions are less and less likely to change, and so form simple models of age-dependent time-reinforcing schemes. These chains, however, exhibit a trichotomy of occupation behaviors depending on parameters.

We show that the average occupation or empirical distribution vector up to time $n$, when variously $0< z< 1$, $z>1$ or $z=1$, converges in probability to a unique ``stationary'' vector $n_G$, converges in law to a nontrivial mixture of point measures, or converges in law to a distribution $m_G$ with no atoms and full support on a simplex respectively, as $n$ tends to infinity. This last type of limit can be interpreted as a sort of ``spreading'' between the cases $0< z < 1$ and $z>1$.

In particular, when $G$ is appropriately chosen, $m_G$ is a Dirichlet distribution, reminiscent of results in Polya urns.