Open Access
VOL. 8 | 2012 Density estimation in high and ultra high dimensions, regularization, and the L1 asymptotics
Anirban DasGupta, S.N. Lahiri

Editor(s) Dominique Fourdrinier, Éric Marchand, Andrew L. Rukhin

Inst. Math. Stat. (IMS) Collect., 2012: 1-23 (2012) DOI: 10.1214/11-IMSCOLL801


This article gives a theoretical treatment of the asymptotics of the L1 error of a model-based estimate of a density f(x|θ) on a finite dimensional Euclidean space ℛk.

The dimension p of the parameter vector θ is considered arbitrary but fixed in Section 2. Two theorems in Section 2 lay out the weak limits of a suitably scaled L1 error, with a general estimating sequence θ̂ and a general family of smooth densities f(x|θ) dominated by some σ-finite measure, the discrete case included. We show that the L1 error converges at the coarsest rate corresponding to the different coordinates of the parameter vector θ. Four applications are detailed in Section 3, a special one being a new confidence interval for a Poisson mean.

Section 4 considers the high and the ultra high dimensional case, where p grows with n. The exact critical growth rate for p when maximum likelihood starts to falter is derived. Maximum likelihood is shown to exhibit a trichotomy of behavior; the desired behavior below the threshold, problematic behavior at the threshold, and disastrous behavior above the threshold.

It is then shown that regularization, if coupled with the right amount of sparsity, can return consistent density estimation, even at the best possible n1/2 rate. We give a complete description of the limiting behavior of the regularized density estimate under different sparsity conditions. Section 4 is specialized to the Gaussian case due to its special importance and well known links to function estimation.


Published: 1 January 2012
First available in Project Euclid: 14 March 2012

zbMATH: 1347.60010

Digital Object Identifier: 10.1214/11-IMSCOLL801

Primary: 60K35 , 60K35
Secondary: 60K35

Keywords: asymptotics , consistency , Density estimation , Gaussian , James-Stein estimator , L_1 error , maximum likelihood , regularization , Sparsity

Rights: Copyright © 2012, Institute of Mathematical Statistics

Back to Top