## The Annals of Statistics

- Ann. Statist.
- Volume 46, Number 6B (2018), 3767-3804.

### Rho-estimators revisited: General theory and applications

Yannick Baraud and Lucien Birgé

#### Abstract

Following Baraud, Birgé and Sart [*Invent. Math.* **207** (2017) 425–517], we pursue our attempt to design a robust universal estimator of the joint distribution of $n$ independent (but not necessarily i.i.d.) observations for an Hellinger-type loss. Given such observations with an unknown joint distribution $\mathbf{P}$ and a dominated model $\mathscr{Q}$ for $\mathbf{P}$, we build an estimator $\widehat{\mathbf{P}}$ based on $\mathscr{Q}$ (a $\rho$-estimator) and measure its risk by an Hellinger-type distance. When $\mathbf{P}$ does belong to the model, this risk is bounded by some quantity which relies on the local complexity of the model in a vicinity of $\mathbf{P}$. In most situations, this bound corresponds to the minimax risk over the model (up to a possible logarithmic factor). When $\mathbf{P}$ does not belong to the model, its risk involves an additional bias term proportional to the distance between $\mathbf{P}$ and $\mathscr{Q}$, whatever the true distribution $\mathbf{P}$. From this point of view, this new version of $\rho$-estimators improves upon the previous one described in Baraud, Birgé and Sart [*Invent. Math.* **207** (2017) 425–517] which required that $\mathbf{P}$ be absolutely continuous with respect to some known reference measure. Further additional improvements have been brought as compared to the former construction. In particular, it provides a very general treatment of the regression framework with random design as well as a computationally tractable procedure for aggregating estimators. We also give some conditions for the maximum likelihood estimator to be a $\rho$-estimator. Finally, we consider the situation where the statistician has at her or his disposal many different models and we build a penalized version of the $\rho$-estimator for model selection and adaptation purposes. In the regression setting, this penalized estimator not only allows one to estimate the regression function but also the distribution of the errors.

#### Article information

**Source**

Ann. Statist., Volume 46, Number 6B (2018), 3767-3804.

**Dates**

Received: June 2016

Revised: November 2017

First available in Project Euclid: 11 September 2018

**Permanent link to this document**

https://projecteuclid.org/euclid.aos/1536631290

**Digital Object Identifier**

doi:10.1214/17-AOS1675

**Mathematical Reviews number (MathSciNet)**

MR3852668

**Zentralblatt MATH identifier**

1407.62169

**Subjects**

Primary: 62G35: Robustness 62G05: Estimation 62G07: Density estimation 62G08: Nonparametric regression 62C20: Minimax procedures 62F99: None of the above, but in this section

**Keywords**

$\rho$-estimation robust estimation density estimation regression with random design statistical models maximum likelihood estimators metric dimension VC-classes

#### Citation

Baraud, Yannick; Birgé, Lucien. Rho-estimators revisited: General theory and applications. Ann. Statist. 46 (2018), no. 6B, 3767--3804. doi:10.1214/17-AOS1675. https://projecteuclid.org/euclid.aos/1536631290

#### Supplemental materials

- Supplement to “Rho-estimators revisited: general theory and applications”. This supplement provides the proofs of most results given in the paper and an additional section (D.10) devoted to robust tests.Digital Object Identifier: doi:10.1214/17-AOS1675SUPPSupplemental files are immediately available to subscribers. Non-subscribers gain access to supplemental files with the purchase of the article.