## The Annals of Applied Probability

- Ann. Appl. Probab.
- Volume 29, Number 4 (2019), 2033-2066.

### Stochastic approximation with random step sizes and urn models with random replacement matrices having finite mean

Ujan Gangopadhyay and Krishanu Maulik

#### Abstract

The stochastic approximation algorithm is a useful technique which has been exploited successfully in probability theory and statistics for a long time. The step sizes used in stochastic approximation are generally taken to be deterministic and same is true for the drift. However, the specific application of urn models with random replacement matrices motivates us to consider stochastic approximation in a setup where both the step sizes and the drift are random, but the sequence is uniformly bounded. The problem becomes interesting when the negligibility conditions on the errors hold only in probability. We first prove a result on stochastic approximation in this setup, which is new in the literature. Then, as an application, we study urn models with random replacement matrices.

In the urn model, the replacement matrices need neither be independent, nor identically distributed. We assume that the replacement matrices are only independent of the color drawn in the same round conditioned on the entire past. We relax the usual second moment assumption on the replacement matrices in the literature and require only first moment to be finite. We require the conditional expectation of the replacement matrix given the past to be close to an irreducible matrix, in an appropriate sense. We do not require any of the matrices to be balanced or nonrandom. We prove convergence of the proportion vector, the composition vector and the count vector in $L^{1}$, and hence in probability. It is to be noted that the related differential equation is of Lotka–Volterra type and can be analyzed directly.

#### Article information

**Source**

Ann. Appl. Probab., Volume 29, Number 4 (2019), 2033-2066.

**Dates**

Received: August 2018

First available in Project Euclid: 23 July 2019

**Permanent link to this document**

https://projecteuclid.org/euclid.aoap/1563869036

**Digital Object Identifier**

doi:10.1214/18-AAP1441

**Mathematical Reviews number (MathSciNet)**

MR3984252

**Zentralblatt MATH identifier**

07120702

**Subjects**

Primary: 62L20: Stochastic approximation

Secondary: 60F15: Strong theorems 60G42: Martingales with discrete parameter

**Keywords**

Urn model random replacement matrix balanced replacement matrix irreducibility stochastic approximation random step size random drift uniform integrability Lotka–Volterra differential equation

#### Citation

Gangopadhyay, Ujan; Maulik, Krishanu. Stochastic approximation with random step sizes and urn models with random replacement matrices having finite mean. Ann. Appl. Probab. 29 (2019), no. 4, 2033--2066. doi:10.1214/18-AAP1441. https://projecteuclid.org/euclid.aoap/1563869036