The Annals of Statistics

Consistent estimation of the basic neighborhood of Markov random fields

Imre Csiszár and Zsolt Talata

Full-text: Open access

Enhanced PDF (238 KB)

Abstract

For Markov random fields on ℤd with finite state space, we address the statistical estimation of the basic neighborhood, the smallest region that determines the conditional distribution at a site on the condition that the values at all other sites are given. A modification of the Bayesian Information Criterion, replacing likelihood by pseudo-likelihood, is proved to provide strongly consistent estimation from observing a realization of the field on increasing finite regions: the estimated basic neighborhood equals the true one eventually almost surely, not assuming any prior bound on the size of the latter. Stationarity of the Markov field is not required, and phase transition does not affect the results.

Article information

Source
Ann. Statist., Volume 34, Number 1 (2006), 123-145.

Dates
First available in Project Euclid: 2 May 2006

Permanent link to this document
https://projecteuclid.org/euclid.aos/1146576258

Digital Object Identifier
doi:10.1214/009053605000000912

Mathematical Reviews number (MathSciNet)
MR2275237

Zentralblatt MATH identifier
1102.62105

Subjects
Primary: 60G60: Random fields 62F12: Asymptotic properties of estimators
Secondary: 62M40: Random fields; image analysis 82B20: Lattice systems (Ising, dimer, Potts, etc.) and systems on graphs

Keywords
Markov random field pseudo-likelihood Gibbs measure model selection information criterion typicality

Citation

Csiszár, Imre; Talata, Zsolt. Consistent estimation of the basic neighborhood of Markov random fields. Ann. Statist. 34 (2006), no. 1, 123--145. doi:10.1214/009053605000000912. https://projecteuclid.org/euclid.aos/1146576258


Export citation

References

  • Akaike, H. (1972). Information theory and an extension of the maximum likelihood principle. In Proc. Second International Symposium on Information Theory. Supplement to Problems of Control and Information Theory (B. N. Petrov and F. Csáki, eds.) 267–281. Akadémiai Kiadó, Budapest.
    Mathematical Reviews (MathSciNet): MR0483125
    Zentralblatt MATH: 0283.62006
  • Azencott, R. (1987). Image analysis and Markov fields. In Proc. First International Conference on Industrial and Applied Mathematics, Paris (J. McKenna and R. Temen, eds.) 53–61. SIAM, Philadelphia.
    Mathematical Reviews (MathSciNet): MR0976851
  • Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems (with discussion). J. Roy. Statist. Soc. Ser. B 36 192–236.
    Mathematical Reviews (MathSciNet): MR0373208
    Zentralblatt MATH: 0327.60067
    Digital Object Identifier: doi:10.1111/j.2517-6161.1974.tb00999.x
  • Besag, J. (1975). Statistical analysis of non-lattice data. The Statistician 24 179–195.
  • Bühlmann, P. and Wyner, A. J. (1999). Variable length Markov chains. Ann. Statist. 27 480–513.
    Mathematical Reviews (MathSciNet): MR1714720
    Digital Object Identifier: doi:10.1214/aos/1018031204
    Project Euclid: euclid.aos/1018031204
  • Comets, F. (1992). On consistency of a class of estimators for exponential families of Markov random fields on the lattice. Ann. Statist. 20 455–468.
    Mathematical Reviews (MathSciNet): MR1150354
    Zentralblatt MATH: 0787.62100
    Digital Object Identifier: doi:10.1214/aos/1176348532
    Project Euclid: euclid.aos/1176348532
  • Csiszár, I. (2002). Large-scale typicality of Markov sample paths and consistency of MDL order estimators. IEEE Trans. Inform. Theory 48 1616–1628.
    Mathematical Reviews (MathSciNet): MR1909476
    Digital Object Identifier: doi:10.1109/TIT.2002.1003842
  • Csiszár, I. and Shields, P. C. (2000). The consistency of the BIC Markov order estimator. Ann. Statist. 28 1601–1619.
    Mathematical Reviews (MathSciNet): MR1835033
    Digital Object Identifier: doi:10.1214/aos/1015957472
    Project Euclid: euclid.aos/1015957472
  • Csiszár, I. and Talata, Zs. (2006). Context tree estimation for not necessarily finite memory processes, via BIC and MDL. IEEE Trans. Inform. Theory 52 1007–1016.
  • Dobrushin, R. L. (1968). The description of a random field by means of conditional probabilities and conditions for its regularity. Theory Probab. Appl. 13 197–224.
    Mathematical Reviews (MathSciNet): MR0231434
    Zentralblatt MATH: 0184.40403
  • Finesso, L. (1992). Estimation of the order of a finite Markov chain. In Recent Advances in Mathematical Theory of Systems, Control, Networks and Signal Processing 1 (H. Kimura and S. Kodama, eds.) 643–645. Mita, Tokyo.
    Mathematical Reviews (MathSciNet): MR1197985
    Zentralblatt MATH: 0784.60071
  • Geman, S. and Graffigne, C. (1987). Markov random field image models and their applications to computer vision. In Proc. International Congress of Mathematicians 2 (A. M. Gleason, ed.) 1496–1517. Amer. Math. Soc., Providence, RI.
    Mathematical Reviews (MathSciNet): MR0934354
    Zentralblatt MATH: 0665.68067
  • Georgii, H. O. (1988). Gibbs Measures and Phase Transitions. de Gruyter, Berlin.
    Mathematical Reviews (MathSciNet): MR0956646
    Zentralblatt MATH: 0657.60122
  • Gidas, B. (1988). Consistency of maximum likelihood and pseudolikelihood estimators for Gibbs distributions. In Stochastic Differential Systems, Stochastic Control Theory and Applications (W. Fleming and P.-L. Lions, eds.) 129–145. Springer, New York.
    Mathematical Reviews (MathSciNet): MR0934721
    Zentralblatt MATH: 0648.62099
  • Gidas, B. (1993). Parameter estimation for Gibbs distributions from fully observed data. In Markov Random Fields: Theory and Application (R. Chellappa and A. Jain, eds.) 471–498. Academic Press, Boston.
    Mathematical Reviews (MathSciNet): MR1214376
  • Hannan, E. J. and Quinn, B. G. (1979). The determination of the order of an autoregression. J. Roy. Statist. Soc. Ser. B 41 190–195.
    Mathematical Reviews (MathSciNet): MR0547244
    Zentralblatt MATH: 0408.62076
    Digital Object Identifier: doi:10.1111/j.2517-6161.1979.tb01072.x
  • Haughton, D. (1988). On the choice of a model to fit data from an exponential family. Ann. Statist. 16 342–355.
    Mathematical Reviews (MathSciNet): MR0924875
    Zentralblatt MATH: 0657.62037
    Digital Object Identifier: doi:10.1214/aos/1176350709
    Project Euclid: euclid.aos/1176350709
  • Ji, C. and Seymour, L. (1996). A consistent model selection procedure for Markov random fields based on penalized pseudolikelihood. Ann. Appl. Probab. 6 423–443.
    Mathematical Reviews (MathSciNet): MR1398052
    Zentralblatt MATH: 0856.62082
    Digital Object Identifier: doi:10.1214/aoap/1034968138
    Project Euclid: euclid.aoap/1034968138
  • Pickard, D. K. (1987). Inference for discrete Markov fields: The simplest non-trivial case. J. Amer. Statist. Assoc. 82 90–96.
    Mathematical Reviews (MathSciNet): MR0883337
    Zentralblatt MATH: 0621.62091
    Digital Object Identifier: doi:10.1080/01621459.1987.10478394
  • Rényi, A. (1970). Probability Theory. North-Holland, Amsterdam.
    Mathematical Reviews (MathSciNet): MR0315747
  • Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461–464.
    Mathematical Reviews (MathSciNet): MR0468014
    Zentralblatt MATH: 0379.62005
    Digital Object Identifier: doi:10.1214/aos/1176344136
    Project Euclid: euclid.aos/1176344136
  • Weinberger, M. J., Rissanen, J. and Feder, M. (1995). A universal finite memory source. IEEE Trans. Inform. Theory 41 643–652.
    Zentralblatt MATH: 0820.94002
    Digital Object Identifier: doi:10.1109/18.382011
  • Willems, F. M. J., Shtarkov, Y. M. and Tjalkens, T. J. (1993). The context-tree weighting method: Basic properties. Technical report, Dept. Electrical Engineering, Eindhoven Univ.
    Zentralblatt MATH: 0837.94011
    Digital Object Identifier: doi:10.1109/18.382012
  • Willems, F. M. J., Shtarkov, Y. M. and Tjalkens, T. J. (2000). Context-tree maximizing. In Proc. 2000 Conf. Information Sciences and Systems TP6-7–TP6-12. Princeton, NJ.

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more