Concentration inequalities for non-causal random fields

Abstract

Concentration inequalities are widely used for analyzing machine learning algorithms. However, the current concentration inequalities cannot be applied to some non-causal processes which appear for instance in Natural Language Processing (NLP). This is mainly due to the non-causal nature of such involved data, in the sense that each data point depends on other neighboring data points. In this paper, we establish a framework for modeling non-causal random fields and prove a Hoeffding-type concentration inequality. The proof of this result is based on a local approximation of the non-causal random field by a function of a finite number of i.i.d. random variables.