The Annals of Statistics

An iterative procedure for general probability measures to obtain I-projections onto intersections of convex sets

Bhaskar Bhattacharya

Full-text: Open access


The iterative proportional fitting procedure (IPFP) was introduced formally by Deming and Stephan in 1940. For bivariate densities, this procedure has been investigated by Kullback and Rüschendorf. It is well known that the IPFP is a sequence of successive I-projections onto sets of probability measures with fixed marginals. However, when finding the I-projection onto the intersection of arbitrary closed, convex sets (e.g., marginal stochastic orders), a sequence of successive I-projections onto these sets may not lead to the actual solution. Addressing this situation, we present a new iterative I-projection algorithm. Under reasonable assumptions and using tools from Fenchel duality, convergence of this algorithm to the true solution is shown. The cases of infinite dimensional IPFP and marginal stochastic orders are worked out in this context.

Article information

Ann. Statist., Volume 34, Number 2 (2006), 878-902.

First available in Project Euclid: 27 June 2006

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62B10: Information-theoretic topics [See also 94A17] 60B10: Convergence of probability measures 65K10: Optimization and variational techniques [See also 49Mxx, 93B40]
Secondary: 60E15: Inequalities; stochastic orderings

Algorithm convergence convex sets Fenchel duality functions I-projection inequality constraints stochastic order


Bhattacharya, Bhaskar. An iterative procedure for general probability measures to obtain I-projections onto intersections of convex sets. Ann. Statist. 34 (2006), no. 2, 878--902. doi:10.1214/009053606000000056.

Export citation


  • Barbu, V. and Precupanu, T. (1986). Convexity and Optimization in Banach Spaces, 2nd ed. Reidel, Dordrecht.
  • Bhattacharya, B. and Dykstra, R. (1995). A general duality approach to I-projections. J. Statist. Plann. Inference 47 203–216.
  • Bhattacharya, B. and Dykstra, R. (1997). A Fenchel duality aspect of iterative I-projection procedures. Ann. Inst. Statist. Math. 49 435–446.
  • Billingsley, P. (1968). Convergence of Probability Measures. Wiley, New York.
  • Cover, T. M. and Thomas, J. (1991). Elements of Information Theory. Wiley, New York.
  • Csiszár, I. (1975). I-divergence geometry of probability distributions and minimization problems. Ann. Probab. 3 146–158.
  • Csiszár, I. (1989). A geometric interpretation of Darroch and Ratcliff's generalized iterative scaling. Ann. Statist. 17 1409–1413.
  • Darroch, J. N. and Ratcliff, D. (1972). Generalized iterative scaling for loglinear models. Ann. Math. Statist. 43 1470–1480.
  • Deming, W. E. and Stephan, F. F. (1940). On a least squares adjustment of a sampled frequency table when the expected marginal totals are known. Ann. Math. Statist. 11 427–444.
  • Dykstra, R. (1985). An iterative procedure for obtaining I-projections onto the intersection of convex sets. Ann. Probab. 13 975–984.
  • Dykstra, R. and Lemke, J. H. (1988). Duality of I-projections and maximum likelihood estimation for log-linear models under cone constraints. J. Amer. Statist. Assoc. 83 546–554.
  • Good, I. J. (1963). Maximum entropy for hypothesis formulation, especially for multidimensional contingency tables. Ann. Math. Statist. 34 911–934.
  • Haberman, S. J. (1974). The Analysis of Frequency Data. Univ. Chicago Press.
  • Haberman, S. J. (1984). Adjustment by minimum discriminant information. Ann. Statist. 12 971–988.
  • Ireland, C. T. and Kullback, S. (1968). Contingency tables with given marginals. Biometrika 55 179–188.
  • Jaynes, E. T. (1957). Information theory and statistical mechanics. Phys. Rev. (2) 106 620–630.
  • Kagan, A. M., Linnik, Y. V. and Rao, C. R. (1973). Characterization Problems in Mathematical Statistics. Wiley, New York.
  • Kruithof, R. (1937). Telefoonverkeersrekening. De Ingenieur 52 E15–E25.
  • Kullback, S. (1959). Information Theory and Statistics. Wiley, New York.
  • Kullback, S. (1968). Probability densities with given marginals. Ann. Math. Statist. 39 1236–1243.
  • Liese, F. (1977). On the existence of $f$-projections. In Topics in Information Theory (I. Csiszár and P. Elias, eds.) 431–446. North-Holland, Amsterdam.
  • Luenberger, D. G. (1969). Optimization by Vector Space Methods. Wiley, New York.
  • Rao, C. R. (1973). Linear Statistical Inference and Its Applications, 2nd ed. Wiley, New York.
  • Robertson, T., Wright, F. T. and Dykstra, R. L. (1988). Order Restricted Statistical Inference. Wiley, New York.
  • Rüschendorf, L. (1995). Convergence of the iterative proportional fitting procedure. Ann. Statist. 23 1160–1174.
  • Rüschendorf, L. and Thomsen, W. (1993). Note on the Schrodinger equation and I-projections. Statist. Probab. Lett. 17 369–375.
  • Sanov, I. N. (1957). On the probability of large deviations of random magnitudes. Mat. Sb. N.S. 42 11–44.
  • Winkler, W. (1990). On Dykstra's iterative fitting procedure. Ann. Probab. 18 1410–1415.