The Annals of Statistics

Consistent estimation of the basic neighborhood of Markov random fields

Imre Csiszár and Zsolt Talata

Full-text: Open access


For Markov random fields on ℤd with finite state space, we address the statistical estimation of the basic neighborhood, the smallest region that determines the conditional distribution at a site on the condition that the values at all other sites are given. A modification of the Bayesian Information Criterion, replacing likelihood by pseudo-likelihood, is proved to provide strongly consistent estimation from observing a realization of the field on increasing finite regions: the estimated basic neighborhood equals the true one eventually almost surely, not assuming any prior bound on the size of the latter. Stationarity of the Markov field is not required, and phase transition does not affect the results.

Article information

Ann. Statist., Volume 34, Number 1 (2006), 123-145.

First available in Project Euclid: 2 May 2006

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60G60: Random fields 62F12: Asymptotic properties of estimators
Secondary: 62M40: Random fields; image analysis 82B20: Lattice systems (Ising, dimer, Potts, etc.) and systems on graphs

Markov random field pseudo-likelihood Gibbs measure model selection information criterion typicality


Csiszár, Imre; Talata, Zsolt. Consistent estimation of the basic neighborhood of Markov random fields. Ann. Statist. 34 (2006), no. 1, 123--145. doi:10.1214/009053605000000912.

Export citation


  • Akaike, H. (1972). Information theory and an extension of the maximum likelihood principle. In Proc. Second International Symposium on Information Theory. Supplement to Problems of Control and Information Theory (B. N. Petrov and F. Csáki, eds.) 267–281. Akadémiai Kiadó, Budapest.
  • Azencott, R. (1987). Image analysis and Markov fields. In Proc. First International Conference on Industrial and Applied Mathematics, Paris (J. McKenna and R. Temen, eds.) 53–61. SIAM, Philadelphia.
  • Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems (with discussion). J. Roy. Statist. Soc. Ser. B 36 192–236.
  • Besag, J. (1975). Statistical analysis of non-lattice data. The Statistician 24 179–195.
  • Bühlmann, P. and Wyner, A. J. (1999). Variable length Markov chains. Ann. Statist. 27 480–513.
  • Comets, F. (1992). On consistency of a class of estimators for exponential families of Markov random fields on the lattice. Ann. Statist. 20 455–468.
  • Csiszár, I. (2002). Large-scale typicality of Markov sample paths and consistency of MDL order estimators. IEEE Trans. Inform. Theory 48 1616–1628.
  • Csiszár, I. and Shields, P. C. (2000). The consistency of the BIC Markov order estimator. Ann. Statist. 28 1601–1619.
  • Csiszár, I. and Talata, Zs. (2006). Context tree estimation for not necessarily finite memory processes, via BIC and MDL. IEEE Trans. Inform. Theory 52 1007–1016.
  • Dobrushin, R. L. (1968). The description of a random field by means of conditional probabilities and conditions for its regularity. Theory Probab. Appl. 13 197–224.
  • Finesso, L. (1992). Estimation of the order of a finite Markov chain. In Recent Advances in Mathematical Theory of Systems, Control, Networks and Signal Processing 1 (H. Kimura and S. Kodama, eds.) 643–645. Mita, Tokyo.
  • Geman, S. and Graffigne, C. (1987). Markov random field image models and their applications to computer vision. In Proc. International Congress of Mathematicians 2 (A. M. Gleason, ed.) 1496–1517. Amer. Math. Soc., Providence, RI.
  • Georgii, H. O. (1988). Gibbs Measures and Phase Transitions. de Gruyter, Berlin.
  • Gidas, B. (1988). Consistency of maximum likelihood and pseudolikelihood estimators for Gibbs distributions. In Stochastic Differential Systems, Stochastic Control Theory and Applications (W. Fleming and P.-L. Lions, eds.) 129–145. Springer, New York.
  • Gidas, B. (1993). Parameter estimation for Gibbs distributions from fully observed data. In Markov Random Fields: Theory and Application (R. Chellappa and A. Jain, eds.) 471–498. Academic Press, Boston.
  • Hannan, E. J. and Quinn, B. G. (1979). The determination of the order of an autoregression. J. Roy. Statist. Soc. Ser. B 41 190–195.
  • Haughton, D. (1988). On the choice of a model to fit data from an exponential family. Ann. Statist. 16 342–355.
  • Ji, C. and Seymour, L. (1996). A consistent model selection procedure for Markov random fields based on penalized pseudolikelihood. Ann. Appl. Probab. 6 423–443.
  • Pickard, D. K. (1987). Inference for discrete Markov fields: The simplest non-trivial case. J. Amer. Statist. Assoc. 82 90–96.
  • Rényi, A. (1970). Probability Theory. North-Holland, Amsterdam.
  • Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461–464.
  • Weinberger, M. J., Rissanen, J. and Feder, M. (1995). A universal finite memory source. IEEE Trans. Inform. Theory 41 643–652.
  • Willems, F. M. J., Shtarkov, Y. M. and Tjalkens, T. J. (1993). The context-tree weighting method: Basic properties. Technical report, Dept. Electrical Engineering, Eindhoven Univ.
  • Willems, F. M. J., Shtarkov, Y. M. and Tjalkens, T. J. (2000). Context-tree maximizing. In Proc. 2000 Conf. Information Sciences and Systems TP6-7–TP6-12. Princeton, NJ.