The success of deep learning has inspired a lot of recent interests in exploiting neural network structures for statistical inference and learning. In this paper, we review some popular deep neural network structures and techniques under the framework of nonparametric regression with measurement errors. In particular, we demonstrate how to use a fully connected feed-forward neural network to approximate the regression function , explain how to use a normalizing flow to approximate the prior distribution of X, and detail how to construct an inference network to generate approximate posterior samples of X. After reviewing recent advances in variational inference for deep neural networks, such as the importance weighted autoencoder, doubly reparametrized gradient estimator, and nonlinear independent components estimation, we describe an inference procedure built upon these advances. An extensive numerical study is presented to compare the neural network approach with classical nonparametric methods, which suggests that the neural network approach is more flexible in accommodating different classes of regression functions and performs superior or comparable to the best available method in many settings.
The author gratefully acknowledge the support of NSF Grants DMS-1943902 to Z Ke, and DMS-1903139 and DMS-2015411 to JS Liu.
The authors thank Editor Sonia Petrone and the anonymous Associate Editor and referees for their great comments, which helped improve the paper.
Zhirui Hu. Zheng Tracy Ke. Jun S. Liu. "Measurement Error Models: From Nonparametric Methods to Deep Neural Networks." Statist. Sci. 37 (4) 473 - 493, November 2022. https://doi.org/10.1214/21-STS834