Incidental Versus Random Nuisance Parameters

Abstract

Let $\{P_{\vartheta,\eta}:(\vartheta, \eta) \in \Theta \times H\}$, with $\Theta \subset \mathbb{R}$ and H arbitrary, be a family of mutually absolutely continuous probability measures on a measurable space $(X, \mathscr{A})$. The problem is to estimate $\vartheta$, based on a sample $(x_1, \cdots, x_n)$ from $\times^n_1 P_{\vartheta,\eta_\nu}$. If $(\eta_1, \cdots, \eta_n)$ are independently distributed according to some unknown prior distribution $\Gamma$, then the distribution of $n^{1/2}(\vartheta^{(n)} - \vartheta)$ under $P^n_{\vartheta,\Gamma}(P_{\vartheta, \Gamma}$ being the $\Gamma$-mixture of $P_{\vartheta,\eta}, \eta \in H$) cannot be more concentrated asymptotically than a certain normal distribution with mean 0, say $N_{(0, \sigma^2_0(\vartheta,\Gamma))}$. Folklore says that such a bound is also valid if $(\eta_1, \cdots, \eta_n)$ are just unknown values of the nuisance parameter: In this case, the distribution cannot be more concentrated asymptotically than $N_{(0, \sigma^2_0(\vartheta,E^{(n)}_{(\eta_1, \cdots, \eta_n)}))}$, where $E^{(n)}_{(\eta_1, \cdots, \eta_n)}$ is the empirical distribution of $(\eta_1,\cdots, \eta_n)$. The purpose of the present paper is to discuss to which extent this conjecture is true. The results are summarized at the end of Sections 1 and 3.