• Bernoulli
  • Volume 23, Number 4A (2017), 2746-2783.

A generalized divergence for statistical inference

Abhik Ghosh, Ian R. Harris, Avijit Maji, Ayanendranath Basu, and Leandro Pardo

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


The power divergence (PD) and the density power divergence (DPD) families have proven to be useful tools in the area of robust inference. In this paper, we consider a superfamily of divergences which contains both of these families as special cases. The role of this superfamily is studied in several statistical applications, and desirable properties are identified and discussed. In many cases, it is observed that the most preferred minimum divergence estimator within the above collection lies outside the class of minimum PD or minimum DPD estimators, indicating that this superfamily has real utility, rather than just being a routine generalization. The limitation of the usual first order influence function as an effective descriptor of the robustness of the estimator is also demonstrated in this connection.

Article information

Bernoulli, Volume 23, Number 4A (2017), 2746-2783.

Received: December 2014
Revised: February 2016
First available in Project Euclid: 9 May 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

breakdown point divergence measure influence function robust estimation $S$-divergence


Ghosh, Abhik; Harris, Ian R.; Maji, Avijit; Basu, Ayanendranath; Pardo, Leandro. A generalized divergence for statistical inference. Bernoulli 23 (2017), no. 4A, 2746--2783. doi:10.3150/16-BEJ826.

Export citation


  • [1] Basu, A., Basu, S. and Chaudhuri, G. (1997). Robust minimum divergence procedures for count data models. Sankhyā Ser. B 59 11–27.
  • [2] Basu, A., Harris, I.R., Hjort, N.L. and Jones, M.C. (1998). Robust and efficient estimation by minimising a density power divergence. Biometrika 85 549–559.
  • [3] Basu, A. and Lindsay, B.G. (1994). Minimum disparity estimation for continuous models: Efficiency, distributions and robustness. Ann. Inst. Statist. Math. 46 683–705.
  • [4] Basu, A., Mandal, A., Martin, N. and Pardo, L. (2013). Testing statistical hypotheses based on the density power divergence. Ann. Inst. Statist. Math. 65 319–348.
  • [5] Basu, A., Mandal, A., Martin, N. and Pardo, L. (2013). Density power divergence tests for composite null hypotheses. Preprint. Available at arXiv:1403.0330.
  • [6] Basu, A. and Sarkar, S. (1994). Minimum disparity estimation in the errors-in-variables model. Statist. Probab. Lett. 20 69–73.
  • [7] Basu, A., Shioya, H. and Park, C. (2011). Statistical Inference: The Minimum Distance Approach. Monographs on Statistics and Applied Probability 120. Boca Raton, FL: CRC Press.
  • [8] Beran, R. (1977). Minimum Hellinger distance estimates for parametric models. Ann. Statist. 5 445–463.
  • [9] Broniatowski, M. and Keziou, A. (2009). Parametric estimation and tests through divergences and the duality technique. J. Multivariate Anal. 100 16–36.
  • [10] Broniatowski, M., Toma, A. and Vajda, I. (2012). Decomposable pseudodistances and applications in statistical estimation. J. Statist. Plann. Inference 142 2574–2585.
  • [11] Brown, M. (1982). Robust line estimation with errors in both variables. J. Amer. Statist. Assoc. 77 71–79.
  • [12] Carroll, R.J. and Gallo, P.P. (1982). Some aspects of robustness in the functional errors-in-variables regression model. Comm. Statist. Theory Methods 11 2573–2585.
  • [13] Cressie, N. and Read, T.R.C. (1984). Multinomial goodness-of-fit tests. J. Roy. Statist. Soc. Ser. B 46 440–464.
  • [14] Csiszár, I. (1963). Eine informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizität von Markoffschen Ketten. Publications of the Mathematical Institute of the Hungarian Academy of Sciences 3 85–107.
  • [15] Dawid, A.P. and Musio, M. (2014). Theory and applications of proper scoring rules. Metron 72 169–183.
  • [16] Dawid, A.P., Musio, M. and Ventura, L. (2016). Minimum scoring rule inference. Scand. J. Stat. 43 123–138.
  • [17] de Boer, P.-T., Kroese, D.P., Mannor, S. and Rubinstein, R.Y. (2005). A tutorial on the cross-entropy method. Ann. Oper. Res. 134 19–67.
  • [18] Donoho, D. and Huber, P.J. (1983). The notion of breakdown point. In A Festschrift for Erich L. Lehmann (P. Bickel, K. Doksum and J. Hodges, Jr., eds.). Wadsworth Statist./Probab. Ser. 157–184. Belmont, CA: Wadsworth.
  • [19] Farcomeni, A. and Greco, L. (2015). Robust Methods for Data Reduction. Londom: Chapman & Hall.
  • [20] Fuller, W.A. (1987). Measurement Error Models. New York: Wiley.
  • [21] Ghosh, A. (2015). Asymptotic properties of minimum $S$-divergence estimator for discrete models. Sankhya A 77 380–407.
  • [22] Ghosh, A. and Basu, A. (2015). The minimum $S$-divergence estimator under continuous models: The Basu–Lindsay approach. Statist. Papers. To appear. DOI:10.1007/s00362-015-0701-3.
  • [23] Ghosh, A. and Basu, A. (2015). Robust estimation for non-homogeneous data and the selection of the optimal tuning parameter: The density power divergence approach. J. Appl. Stat. 42 2056–2072.
  • [24] Ghosh, A. and Basu, A. (2016). Testing composite null hypotheses based on $S$-divergences. Statist. Probab. Lett. 114 38–47.
  • [25] Ghosh, A., Basu, A. and Pardo, L. (2016). On the robustness of a divergence based test of simple statistical hypotheses. J. Statist. Plann. Inference 161 91–108.
  • [26] Ghosh, A., Harris, I.R., Maji, A., Basu, A. and Pardo, L. (2013). A Generalized Divergence for Statistical Inference. Technical Report, BIRU/2013/3, Bayesian and Interdisiplinary Research Unit. Kolkata, India: Indian Statistical Institute.
  • [27] Ghosh, A., Harris, I.R., Maji, A., Basu, A. and Pardo, L. (2016). Supplement to “A generalized divergence for statistical inference.” DOI:10.3150/16-BEJ826SUPP.
  • [28] Ghosh, A., Maji, A. and Basu, A. (2013). Robust Inference Based on Divergences in Reliability Systems. In Applied Reliability Engineering and Risk Analysis. Probabilistic Models and Statistical Inference (I. Frenkel, A. Karagrigoriou, A. Lisnianski and A. Kleyner, eds.). New York: Wiley.
  • [29] Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J. and Stahel, W.A. (1986). Robust Statistics: The Approach Based on Influence Functions. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. New York: Wiley.
  • [30] Heritier, S. and Ronchetti, E. (1994). Robust bounded-influence tests in general parametric models. J. Amer. Statist. Assoc. 89 897–904.
  • [31] Hong, C. and Kim, Y. (2001). Automatic selection of the tuning parameter in the minimum density power divergence estimation. J. Korean Statist. Soc. 30 453–465.
  • [32] Jiménez, R. and Shao, Y. (2001). On robustness and efficiency of minimum divergence estimators. TEST 10 241–248.
  • [33] Lindsay, B.G. (1994). Efficiency versus robustness: The case for minimum Hellinger distance and related methods. Ann. Statist. 22 1081–1114.
  • [34] Mameli, V. and Ventura, L. (2015). Higher-order asymptotics for scoring rules. J. Statist. Plann. Inference 165 13–26.
  • [35] Morales, D., Pardo, L. and Vajda, I. (1995). Asymptotic divergence of estimates of discrete distributions. J. Statist. Plann. Inference 48 347–369.
  • [36] Pardo, L. (2006). Statistical Inference Based on Divergence Measures. Statistics: Textbooks and Monographs 185. Boca Raton, FL: Chapman & Hall/CRC.
  • [37] Patra, S., Maji, A., Basu, A. and Pardo, L. (2013). The power divergence and the density power divergence families: The mathematical connection. Sankhya B 75 16–28.
  • [38] Read, T.R.C. and Cressie, N.A.C. (1988). Goodness-of-Fit Statistics for Discrete Multivariate Data. New York: Springer.
  • [39] Scott, D.W. (2001). Parametric statistical modeling by minimum integrated square error. Technometrics 43 274–285.
  • [40] Shore, J.E. and Johnson, R.W. (1980). Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE Trans. Inform. Theory 26 26–37.
  • [41] Simpson, D.G. (1987). Minimum Hellinger distance estimation for the analysis of count data. J. Amer. Statist. Assoc. 82 802–807.
  • [42] Simpson, D.G. (1989). Hellinger deviance tests: Efficiency, breakdown points, and examples. J. Amer. Statist. Assoc. 84 107–113.
  • [43] Stigler, S.M. (1977). Do robust estimators work with real data? Ann. Statist. 5 1055–1098.
  • [44] Tamura, R.N. and Boos, D.D. (1986). Minimum Hellinger distance estimation for multivariate location and covariance. J. Amer. Statist. Assoc. 81 223–229.
  • [45] Toma, A. and Broniatowski, M. (2011). Dual divergence estimators and tests: Robustness results. J. Multivariate Anal. 102 20–36.
  • [46] Toma, A. and Leoni-Aubin, S. (2015). Robust portfolio optimization using pseudodistances. PLoS ONE 10 e0140546. DOI:10.1371/journal.pone.0140546.
  • [47] Vajda, I. (1989). Theory of Statistical Inference and Information. Dordrecht: Kluwer Academic.
  • [48] Warwick, J. and Jones, M.C. (2005). Choosing a robustness tuning parameter. J. Stat. Comput. Simul. 75 581–588.
  • [49] Wittenberg, M. (1962). An introduction to maximum entropy and minimum cross-entropy estimation using stata. Stata J. 10 315–330.
  • [50] Zamar, R.H. (1989). Robust estimation in the errors-in-variables model. Biometrika 76 149–160.

Supplemental materials

  • Supplement to “A generalized divergence for statistical inference”. Supplement contains all the assumptions required for the asymptotic derivations and proofs of all the technical results presented in the paper. It also contains some remarks on the computation of MSDE and a simulation study illustrating the performance of the MSHDE under the bivariate normal model.