We study the asymptotic optimal control of multi-class restless bandits. A restless bandit is a controllable stochastic process whose state evolution depends on whether or not the bandit is made active. Since finding the optimal control is typically intractable, we propose a class of priority policies that are proved to be asymptotically optimal under a global attractor property and a technical condition. We consider both a fixed population of bandits as well as a dynamic population where bandits can depart and arrive. As an example of a dynamic population of bandits, we analyze a multi-class $\mathit{M/M/S+M}$ queue for which we show asymptotic optimality of an index policy.

We combine fluid-scaling techniques with linear programming results to prove that when bandits are indexable, Whittle’s index policy is included in our class of priority policies. We thereby generalize a result of Weber and Weiss [J. Appl. Probab. 27 (1990) 637–648] about asymptotic optimality of Whittle’s index policy to settings with (i) several classes of bandits, (ii) arrivals of new bandits and (iii) multiple actions.

Indexability of the bandits is not required for our results to hold. For nonindexable bandits, we describe how to select priority policies from the class of asymptotically optimal policies and present numerical evidence that, outside the asymptotic regime, the performance of our proposed priority policies is nearly optimal.

This paper develops a new methodology for studying continuous-time Nash equilibrium in a financial market with asymmetrically informed agents. This approach allows us to lift the restriction of risk neutrality imposed on market makers by the current literature. It turns out that, when the market makers are risk averse, the optimal strategies of the agents are solutions of a forward–backward system of partial and stochastic differential equations. In particular, the price set by the market makers solves a nonstandard “quadratic” backward stochastic differential equation. The main result of the paper is the existence of a Markovian solution to this forward–backward system on an arbitrary time interval, which is obtained via a fixed-point argument on the space of absolutely continuous distribution functions. Moreover, the equilibrium obtained in this paper is able to explain several stylized facts which are not captured by the current asymmetric information models.

We prove Airy process variational formulas for the one-point probability distribution of (discrete time parallel update) TASEP with general initial data, as well as last passage percolation from a general down-right lattice path to a point. We also consider variants of last passage percolation with inhomogeneous parameter geometric weights and provide variational formulas of a similar nature. This proves one aspect of the conjectural description of the renormalization fixed point of the Kardar–Parisi–Zhang universality class.

A new class of explicit Euler schemes, which approximate stochastic differential equations (SDEs) with superlinearly growing drift and diffusion coefficients, is proposed in this article. It is shown, under very mild conditions, that these explicit schemes converge in probability and in $\mathcal{L}^{p}$ to the solution of the corresponding SDEs. Moreover, rate of convergence estimates are provided for $\mathcal{L}^{p}$ and almost sure convergence. In particular, the strong order $1/2$ is recovered in the case of uniform $\mathcal{L}^{p}$-convergence.

We give a general Gaussian bound for the first chaos (or innovation) of point processes with stochastic intensity constructed by embedding in a bivariate Poisson process. We apply the general result to nonlinear Hawkes processes, providing quantitative central limit theorems.

We construct a stationary ergodic process $X_{1},X_{2},\ldots$ such that each $X_{t}$ has the uniform distribution on the unit square and the length $L_{n}$ of the shortest path through the points $X_{1},X_{2},\ldots,X_{n}$ is not asymptotic to a constant times the square root of $n$. In other words, we show that the Beardwood, Halton, and Hammersley [Proc. Cambridge Philos. Soc. 55 (1959) 299–327] theorem does not extend from the case of independent uniformly distributed random variables to the case of stationary ergodic sequences with uniform marginal distributions.

We consider the classical estimation problem of an unknown drift parameter within classes of nondegenerate diffusion processes. Using rough path theory (in the sense of T. Lyons), we analyze the Maximum Likelihood Estimator (MLE) with regard to its pathwise stability properties as well as robustness toward misspecification in volatility and even the very nature of the noise. Two numerical examples demonstrate the practical relevance of our results.

We consider certain respondent-driven sampling procedures on dense graphs. We show that if the sequence of the vertex-sets is ergodic then the limiting graph can be expressed in terms of the original dense graph via a transformation related to the invariant measure of the ergodic sequence. For specific sampling procedures, we describe the transformation explicitly.

We consider the problem of reconstructing sparse symmetric block models with two blocks and connection probabilities $a/n$ and $b/n$ for inter- and intra-block edge probabilities, respectively. It was recently shown that one can do better than a random guess if and only if $(a-b)^{2}>2(a+b)$. Using a variant of belief propagation, we give a reconstruction algorithm that is optimal in the sense that if $(a-b)^{2}>C(a+b)$ for some constant $C$ then our algorithm maximizes the fraction of the nodes labeled correctly. Ours is the only algorithm proven to achieve the optimal fraction of nodes labeled correctly. Along the way, we prove some results of independent interest regarding robust reconstruction for the Ising model on regular and Poisson trees.

Random sampling of large Markov matrices with a tunable spectral gap, a nonuniform stationary distribution and a nondegenerate limiting empirical spectral distribution (ESD) is useful. Fix $c>0$ and $p>0$. Let $A_{n}$ be the adjacency matrix of a random graph following $\mathrm{G}(n,p/n)$, known as the Erdős–Rényi distribution. Add $c/n$ to each entry of $A_{n}$ and then normalize its rows. It is shown that the resulting Markov matrix has the desired properties. Its ESD weakly converges in probability to a symmetric nondegenerate distribution, and its extremal eigenvalues, other than 1, fall in $[-1/\sqrt{1+c/k},-b]\cup[b,1/\sqrt{1+c/k}]$ for any $0<b<1/\sqrt{1+c}$, where $k=\lfloor p\rfloor+1$. Thus, for $p\in(0,1)$, the spectral gap tends to $1-1/\sqrt{1+c}$.

Given the univariate marginals of a real-valued, continuous-time martingale, (resp., a family of measures parameterised by $t\in[0,T]$ which is increasing in convex order, or a double continuum of call prices), we construct a family of pure-jump martingales which mimic that martingale (resp., are consistent with the family of measures, or call prices). As an example, we construct a fake Brownian motion. Then, under a further “dispersion” assumption, we construct the martingale which (within the family of martingales which are consistent with a given set of measures) has the smallest expected total variation. We also give a pathwise inequality, which in the mathematical finance context yields a model-independent sub-hedge for an exotic security with payoff equal to the total variation of the price process.

We introduce new integrable exclusion and zero-range processes on the one-dimensional lattice that generalize the $q$-Hahn TASEP and the $q$-Hahn Boson (zero-range) process introduced in [J. Phys. A 46 (2013) 465205, 25] and further studied in [Int. Math. Res. Not. IMRN 14 (2015) 5577–5603], by allowing jumps in both directions. Owing to a Markov duality, we prove moment formulas for the locations of particles in the exclusion process. This leads to a Fredholm determinant formula that characterizes the distribution of the location of any particle. We show that the model-dependent constants that arise in the limit theorems predicted by the KPZ scaling theory are recovered by a steepest descent analysis of the Fredholm determinant. For some choice of the parameters, our model specializes to the multi-particle-asymmetric diffusion model introduced in [Phys. Rev. E 58 (1998) 4181]. In that case, we make a precise asymptotic analysis that confirms KPZ universality predictions. Surprisingly, we also prove that in the partially asymmetric case, the location of the first particle also enjoys cube-root fluctuations which follow Tracy–Widom GUE statistics.

This paper deals with a parametrized family of partially observed bivariate Markov chains. We establish that, under very mild assumptions, the limit of the normalized log-likelihood function is maximized when the parameters belong to the equivalence class of the true parameter, which is a key feature for obtaining the consistency of the maximum likelihood estimators (MLEs) in well-specified models. This result is obtained in the general framework of partially dominated models. We examine two specific cases of interest, namely, hidden Markov models (HMMs) and observation-driven time series models. In contrast with previous approaches, the identifiability is addressed by relying on the uniqueness of the invariant distribution of the Markov chain associated to the complete data, regardless its rate of convergence to the equilibrium.

As an application of Stein’s method for Poisson approximation, we prove rates of convergence for the tail probabilities of two scan statistics that have been suggested for detecting local signals in sequences of independent random variables subject to possible change-points. Our formulation deals simultaneously with ordinary and with large deviations.

We study competing first passage percolation on graphs generated by the configuration model. At time 0, vertex 1 and vertex 2 are infected with the type 1 and the type 2 infection, respectively, and an uninfected vertex then becomes type 1 (2) infected at rate $\lambda_{1}$ ($\lambda_{2}$) times the number of edges connecting it to a type 1 (2) infected neighbor. Our main result is that, if the degree distribution is a power-law with exponent $\tau\in(2,3)$, then as the number of vertices tends to infinity and with high probability, one of the infection types will occupy all but a finite number of vertices. Furthermore, which one of the infections wins is random and both infections have a positive probability of winning regardless of the values of $\lambda_{1}$ and $\lambda_{2}$. The picture is similar with multiple starting points for the infections.

In this paper, we find a series of equalities which characterize the symmetry of the forming times of a family of similar cycles for discrete-time and continuous-time Markov chains. Moreover, we use these cycle symmetries to study the circulation fluctuations for Markov chains. We prove that the sample circulations along a family of cycles passing through a common state satisfy a large deviation principle with a rate function which has a highly nonobvious symmetry. Further extensions and applications to statistical physics and biochemistry are also discussed, especially the fluctuation theorems for the sample net circulations.

We introduce a class of reinforcement models where, at each time step $t$, one first chooses a random subset $A_{t}$ of colours (independently of the past) from $n$ colours of balls, and then chooses a colour $i$ from this subset with probability proportional to the number of balls of colour $i$ in the urn raised to the power $\alpha>1$. We consider stability of equilibria for such models and establish the existence of phase transitions in a number of examples, including when the colours are the edges of a graph; a context which is a toy model for the formation and reinforcement of neural connections. We conjecture that for any graph $G$ and all $\alpha$ sufficiently large, the set of stable equilibria is supported on so-called whisker-forests, which are forests whose components have diameter between 1 and 3.

From its beginning, there have been attempts by physicists to formulate quantum mechanics without requiring the use of wave functions. An interesting recent approach takes the point of view that quantum effects arise solely from the interaction of finitely many classical “worlds.” The wave function is then recovered (as a secondary object) from observations of particles in these worlds, without knowing the world from which any particular observation originates. Hall, Deckert and Wiseman [Phys. Rev. X 4 (2014) 041013] have introduced an explicit many-interacting-worlds harmonic oscillator model to provide support for this approach. In this note, we provide a proof of their claim that the particle configuration is asymptotically Gaussian, thus matching the stationary ground-state solution of Schrödinger’s equation when the number of worlds goes to infinity. We also construct a Markov chain based on resampling from the particle configuration and show that it converges to an Ornstein–Uhlenbeck process, matching the time-dependent solution as well.

We are interested in the asymptotic behavior of Markov chains on the set of positive integers for which, loosely speaking, large jumps are rare and occur at a rate that behaves like a negative power of the current state, and such that small positive and negative steps of the chain roughly compensate each other. If $X_{n}$ is such a Markov chain started at $n$, we establish a limit theorem for $\frac{1}{n}X_{n}$ appropriately scaled in time, where the scaling limit is given by a nonnegative self-similar Markov process. We also study the asymptotic behavior of the time needed by $X_{n}$ to reach some fixed finite set. We identify three different regimes (roughly speaking the transient, the recurrent and the positive-recurrent regimes) in which $X_{n}$ exhibits different behavior. The present results extend those of Haas and Miermont [Bernoulli 17 (2011) 1217–1247] who focused on the case of nonincreasing Markov chains. We further present a number of applications to the study of Markov chains with asymptotically zero drifts such as Bessel-type random walks, nonnegative self-similar Markov processes, invariance principles for random walks conditioned to stay positive and exchangeable coalescence-fragmentation processes.