The Annals of Statistics
- Ann. Statist.
- Volume 19, Number 3 (1991), 1347-1369.
Approximation of Density Functions by Sequences of Exponential Families
Probability density functions are estimated by the method of maximum likelihood in sequences of regular exponential families. This method is also familiar as entropy maximization subject to empirical constraints. The approximating families of log-densities that we consider are polynomials, splines and trigonometric series. Bounds on the relative entropy (Kullback-Leibler distance) between the true density and the estimator are obtained and rates of convergence are established for log-density functions assumed to have square integrable derivatives.
Ann. Statist., Volume 19, Number 3 (1991), 1347-1369.
First available in Project Euclid: 12 April 2007
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 62G05: Estimation
Secondary: 41A17: Inequalities in approximation (Bernstein, Jackson, Nikol s kii-type inequalities) 62B10: Information-theoretic topics [See also 94A17] 62F12: Asymptotic properties of estimators
Barron, Andrew R.; Sheu, Chyong-Hwa. Approximation of Density Functions by Sequences of Exponential Families. Ann. Statist. 19 (1991), no. 3, 1347--1369. doi:10.1214/aos/1176348252. https://projecteuclid.org/euclid.aos/1176348252
- See Correction: Andrew R. Barron, Chyong-Hwa Sheu. Correction: Approximation of Density Functions by Sequences of Exponential Families. Ann. Statist., Volume 19, Number 4 (1991), 2284--2284.