Asymptotic Suprema of Averaged Random Functions

Abstract

Suppose $X_i$ are i.i.d. random variables taking values in $\mathscr{X}, \Theta$ is a parameter space and $y: \mathscr{X} \times \Theta \rightarrow \mathbf{R}$ is a map. Define the averages $S_n(y, \theta) = (1/n)\sum^n_{i = 1}y(X_i, \theta)$ and the truncated expectations $T_m(y, \theta) = \mathbf{E} \max(y(X_1, \theta), - m)$. Under the hypothesis of global dominance [i.e., $\mathbf{E} \sup_\Theta y(X_1, \theta) < \infty$] and some regularity conditions, the main result of the paper characterizes the asymptotic suprema of $S_n$ as follows. For any subset $G$ of $\Theta$, with probability 1, $\lim_{n \rightarrow \infty} \sup_{\theta \in G} S_N(y, \theta) = \lim_{m \rightarrow \infty} \sup_{\theta \in G} T_m(y, \theta).$ This has immediate application to consistency of $M$-estimators. In particular, under global dominance, maxima of $S_n$ must converge to the same limit as the maxima of $T_m(y, \theta)$ almost surely. We also obtain necessary and sufficient conditions for consistency resembling Huber's in the case of local dominance [i.e., each $\theta \in \Theta$ has a neighborhood $N(\theta)$ such that $\mathbf{E} \sup_{\psi \in N(\theta)}y(X_1, \psi) < \infty\rbrack.$ In this case there must exist a function $b(\theta) \geq 1$ such that $y/b$ is globally dominated and maxima of $T_m(y/b,\theta)$ converge.