Open Access
June 2003 Likelihood ratio of unidentifiable models and multilayer neural networks
Kenji Fukumizu
Ann. Statist. 31(3): 833-851 (June 2003). DOI: 10.1214/aos/1056562464

Abstract

This paper discusses the behavior of the maximum likelihood estimator (MLE), in the case that the true parameter cannot be identified uniquely. Among many statistical models with unidentifiability, neural network models are the main concern of this paper. It is known in some models with unidentifiability that the asymptotics of the likelihood ratio of the MLE has an unusually larger order. Using the framework of locally conic models put forth by Dacunha-Castelle and Gassiat as a generalization of Hartigan's idea, a useful sufficient condition of such larger orders is derived. This result is applied to neural network models, and a larger order is proved if the true function is given by a smaller model. Also, under the condition that the model has at least two redundant hidden units, a log n lower bound for the likelihood ratio is derived.

Citation

Download Citation

Kenji Fukumizu. "Likelihood ratio of unidentifiable models and multilayer neural networks." Ann. Statist. 31 (3) 833 - 851, June 2003. https://doi.org/10.1214/aos/1056562464

Information

Published: June 2003
First available in Project Euclid: 25 June 2003

zbMATH: 1032.62020
MathSciNet: MR1994732
Digital Object Identifier: 10.1214/aos/1056562464

Subjects:
Primary: 62F12
Secondary: 62F10

Keywords: likelihood ratio , locally conic model , multilayer neural networks , unidentifiable model

Rights: Copyright © 2003 Institute of Mathematical Statistics

Vol.31 • No. 3 • June 2003
Back to Top