A New Dynamic Stochastic Approximation Procedure

Abstract

This paper considers Robbins-Monro stochastic approximation when the regression function changes with time. At time $n$, one can select $X_n$ and observe an unbiased estimator of the regression function evaluated at $X_n$. Let $\theta_n$ be the root of the regression function at time $n$. Our goal is to select the sequence $X_n$ so that $X_n - \theta_n$ converges to 0. It is assumed that $\theta_n = f(s_n)$ for $s_n$ known at time $n$ and $f$ an unknown element of a class of functions. Under certain conditions on this class and on the sequence of regression functions, we obtain a random sequence $X_n$ such that $|X_n - \theta_n|$ converges to 0 in Cesaro mean with probability 1. Under more stringent conditions, $X_n - \theta_n$ converges to 0 with probability 1.