The Annals of Statistics

Probability Inequalities for Likelihood Ratios and Convergence Rates of Sieve MLES

Wing Hung Wong and Xiaotong Shen

Full-text: Open access


Let $Y_1,\ldots, Y_n$ be independent identically distributed with density $p_0$ and let $\mathscr{F}$ be a space of densities. We show that the supremum of the likelihood ratios $\prod^n_{i=1} p(Y_i)/p_0(Y_i)$, where the supremum is over $p \in \mathscr{F}$ with $\|p^{1/2} - p^{1/2}_0\|_2 \geq \varepsilon$, is exponentially small with probability exponentially close to 1. The exponent is proportional to $n\varepsilon^2$. The only condition required for this to hold is that $\varepsilon$ exceeds a value determined by the bracketing Hellinger entropy of $\mathscr{F}$. A similar inequality also holds if we replace $\mathscr{F}$ by $\mathscr{F}_n$ and $p_0$ by $q_n$, where $q_n$ is an approximation to $p_0$ in a suitable sense. These results are applied to establish rates of convergence of sieve MLEs. Furthermore, weak conditions are given under which the "optimal" rate $\varepsilon_n$ defined by $H(\varepsilon_n, \mathscr{F}) = n\varepsilon^2_n$, where $H(\cdot, \mathscr{F})$ is the Hellinger entropy of $\mathscr{F}$, is nearly achievable by sieve estimators.

Article information

Ann. Statist., Volume 23, Number 2 (1995), 339-362.

First available in Project Euclid: 11 April 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier


Primary: 62A10
Secondary: 62F12: Asymptotic properties of estimators 62G20: Asymptotic properties

Hellinger distance bracketing metric entropy Kullback-Leibler number exponential inequality


Wong, Wing Hung; Shen, Xiaotong. Probability Inequalities for Likelihood Ratios and Convergence Rates of Sieve MLES. Ann. Statist. 23 (1995), no. 2, 339--362. doi:10.1214/aos/1176324524.

Export citation