This article gives a theoretical treatment of the asymptotics of the L1 error of a model-based estimate of a density f(x|θ) on a finite dimensional Euclidean space ℛk.
The dimension p of the parameter vector θ is considered arbitrary but fixed in Section 2. Two theorems in Section 2 lay out the weak limits of a suitably scaled L1 error, with a general estimating sequence θ̂ and a general family of smooth densities f(x|θ) dominated by some σ-finite measure, the discrete case included. We show that the L1 error converges at the coarsest rate corresponding to the different coordinates of the parameter vector θ. Four applications are detailed in Section 3, a special one being a new confidence interval for a Poisson mean.
Section 4 considers the high and the ultra high dimensional case, where p grows with n. The exact critical growth rate for p when maximum likelihood starts to falter is derived. Maximum likelihood is shown to exhibit a trichotomy of behavior; the desired behavior below the threshold, problematic behavior at the threshold, and disastrous behavior above the threshold.
It is then shown that regularization, if coupled with the right amount of sparsity, can return consistent density estimation, even at the best possible n−1/2 rate. We give a complete description of the limiting behavior of the regularized density estimate under different sparsity conditions. Section 4 is specialized to the Gaussian case due to its special importance and well known links to function estimation.
Digital Object Identifier: 10.1214/11-IMSCOLL801