Estimation of smooth functionals in normal models: Bias reduction and asymptotic efficiency

Abstract

Let ${\mathit{X}}_1,\dots ,{\mathit{X}}_{\mathit{n}}$ be i.i.d. random variables sampled from a normal distribution $\mathit{N}(\mathit{\mu },\mathrm{\Sigma })$ in ${\mathbb{R}}^{\mathit{d}}$ with unknown parameter $\mathit{\theta }=(\mathit{\mu },\mathrm{\Sigma })\in \mathrm{\Theta }:={\mathbb{R}}^{\mathit{d}}\times {\mathcal{C}}_{+}^{\mathit{d}}$, where ${\mathcal{C}}_{+}^{\mathit{d}}$ is the cone of positively definite covariance operators in ${\mathbb{R}}^{\mathit{d}}$. Given a smooth functional $\mathit{f}:\mathrm{\Theta }\mapsto {\mathbb{R}}^1$, the goal is to estimate $\mathit{f}(\mathit{\theta })$ based on ${\mathit{X}}_1,\dots ,{\mathit{X}}_{\mathit{n}}$. Let

$$\mathrm{\Theta }(\mathit{a};\mathit{d}):={\mathbb{R}}^{\mathit{d}}\times \{\mathrm{\Sigma }\in {\mathcal{C}}_{+}^{\mathit{d}}:\mathit{\sigma }(\mathrm{\Sigma })\subset [1/\mathit{a},\mathit{a}]\},\mathit{a}\ge 1,$$

where $\mathit{\sigma }(\mathrm{\Sigma })$ is the spectrum of covariance Σ. Let $\hat{\mathit{\theta }}:=(\hat{\mathit{\mu }},\hat{\mathrm{\Sigma }})$, where $\hat{\mathit{\mu }}$ is the sample mean and $\hat{\mathrm{\Sigma }}$ is the sample covariance, based on the observations ${\mathit{X}}_1,\dots ,{\mathit{X}}_{\mathit{n}}$. For an arbitrary functional $\mathit{f}\in {\mathit{C}}^{\mathit{s}}(\mathrm{\Theta })$, $\mathit{s}=\mathit{k}+1+\mathit{\rho },\mathit{k}\ge 0,\mathit{\rho }\in (0,1]$, we define a functional ${\mathit{f}}_{\mathit{k}}:\mathrm{\Theta }\mapsto \mathbb{R}$ such that

$$\underset{\mathit{\theta }\in \mathrm{\Theta }(\mathit{a};\mathit{d})}{sup}{\Vert {\mathit{f}}_{\mathit{k}}(\hat{\mathit{\theta }})-\mathit{f}(\mathit{\theta })\Vert }_{{\mathit{L}}_2({\mathbb{P}}_{\mathit{\theta }})}{\lesssim }_{\mathit{s},\mathit{\beta }}\Vert \mathit{f}{\Vert }_{{\mathit{C}}^{\mathit{s}}(\mathrm{\Theta })}\left[\right(\frac{\mathit{a}}{\sqrt{\mathit{n}}}\vee {\mathit{a}}^{\mathit{\beta }\mathit{s}}{\left(\sqrt{\frac{\mathit{d}}{\mathit{n}}}\right)}^{\mathit{s}})\wedge 1],$$

where $\mathit{\beta }=1$ for $\mathit{k}=0$ and $\mathit{\beta }>\mathit{s}-1$ is arbitrary for $\mathit{k}\ge 1$. This error rate is minimax optimal and similar bounds hold for more general loss functions. If $\mathit{d}={\mathit{d}}_{\mathit{n}}\le {\mathit{n}}^{\mathit{\alpha }}$ for some $\mathit{\alpha }\in (0,1)$ and $\mathit{s}\ge \frac{1}{1-\mathit{\alpha }}$, the rate becomes $\mathit{O}({\mathit{n}}^{-1/2})$. Moreover, for $\mathit{s}>\frac{1}{1-\mathit{\alpha }}$, the estimator ${\mathit{f}}_{\mathit{k}}(\hat{\mathit{\theta }})$ is shown to be asymptotically efficient. The crucial part of the construction of estimator ${\mathit{f}}_{\mathit{k}}(\hat{\mathit{\theta }})$ is a bias reduction method studied in the paper for more general statistical models than normal.