The Annals of Statistics

Quasi-Bayesian analysis of nonparametric instrumental variables models

Kengo Kato

Full-text: Open access


This paper aims at developing a quasi-Bayesian analysis of the nonparametric instrumental variables model, with a focus on the asymptotic properties of quasi-posterior distributions. In this paper, instead of assuming a distributional assumption on the data generating process, we consider a quasi-likelihood induced from the conditional moment restriction, and put priors on the function-valued parameter. We call the resulting posterior quasi-posterior, which corresponds to “Gibbs posterior” in the literature. Here we focus on priors constructed on slowly growing finite-dimensional sieves. We derive rates of contraction and a nonparametric Bernstein–von Mises type result for the quasi-posterior distribution, and rates of convergence for the quasi-Bayes estimator defined by the posterior expectation. We show that, with priors suitably chosen, the quasi-posterior distribution (the quasi-Bayes estimator) attains the minimax optimal rate of contraction (convergence, resp.). These results greatly sharpen the previous related work.

Article information

Ann. Statist., Volume 41, Number 5 (2013), 2359-2390.

First available in Project Euclid: 5 November 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression 62G20: Asymptotic properties

Asymptotic normality inverse problem nonparametric instrumental variables model quasi-Bayes rates of contraction


Kato, Kengo. Quasi-Bayesian analysis of nonparametric instrumental variables models. Ann. Statist. 41 (2013), no. 5, 2359--2390. doi:10.1214/13-AOS1150.

Export citation


  • [1] Agapiou, S., Larsson, S. and Stuart, A. M. (2013). Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems. Stochastic Process. Appl. 123 3828–3860.
  • [2] Ai, C. and Chen, X. (2003). Efficient estimation of models with conditional moment restrictions containing unknown functions. Econometrica 71 1795–1843.
  • [3] Belloni, A. and Chernozhukov, V. (2009). Posterior inference in curved exponential families under increasing dimensions. Preprint. Available at arXiv:0904.3132.
  • [4] Belloni, A. and Chernozhukov, V. (2009). On the computational complexity of MCMC-based estimators in large samples. Ann. Statist. 37 2011–2055.
  • [5] Blundell, R., Chen, X. and Kristensen, D. (2007). Semi-nonparametric IV estimation of shape-invariant Engel curves. Econometrica 75 1613–1669.
  • [6] Bontemps, D. (2011). Bernstein–von Mises theorems for Gaussian regression with increasing number of regressors. Ann. Statist. 39 2557–2584.
  • [7] Boucheron, S. and Gassiat, E. (2009). A Bernstein–von Mises theorem for discrete probability distributions. Electron. J. Stat. 3 114–148.
  • [8] Carrasco, M., Florens, J. P. and Renault, E. (2007). Linear inverse problems in structural econometrics: Estimation based on spectral decomposition and regularization. In Handbook of Econometrics, Vol. 6 (J. J. Heckman and E. E. Leamer, eds.) 5633–5751. Elsevier, Amsterdam.
  • [9] Cavalier, L. (2008). Nonparametric statistical inverse problems. Inverse Problems 24 034004, 19.
  • [10] Chen, X. and Pouzo, D. (2012). Estimation of nonparametric conditional moment models with possibly nonsmooth generalized residuals. Econometrica 80 277–321.
  • [11] Chen, X. and Reiss, M. (2011). On rate optimality for ill-posed inverse problems in econometrics. Econometric Theory 27 497–521.
  • [12] Chernozhukov, V. and Hong, H. (2003). An MCMC approach to classical estimation. J. Econometrics 115 293–346.
  • [13] Clarke, B. and Ghosal, S. (2010). Reference priors for exponential families with increasing dimension. Electron. J. Stat. 4 737–780.
  • [14] Cohen, A., Daubechies, I. and Vial, P. (1993). Wavelets on the interval and fast wavelet transforms. Appl. Comput. Harmon. Anal. 1 54–81.
  • [15] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903–923.
  • [16] Darolles, S., Fan, Y., Florens, J. P. and Renault, E. (2011). Nonparametric instrumental regression. Econometrica 79 1541–1565.
  • [17] D’Haultfoeuille, X. (2011). On the completeness condition in nonparametric instrumental problems. Econometric Theory 27 460–471.
  • [18] Florens, J. P. and Simoni, A. (2010). Regularized priors for linear inverse problems. IDEI Working Paper 767-2013.
  • [19] Florens, J.-P. and Simoni, A. (2012). Regularized posteriors in linear ill-posed inverse problems. Scand. J. Stat. 39 214–235.
  • [20] Florens, J.-P. and Simoni, A. (2012). Nonparametric estimation of an instrumental regression: A quasi-Bayesian approach based on regularized posterior. J. Econometrics 170 458–475.
  • [21] Gagliardini, P. and Scaillet, O. (2012). Nonparametric instrumental variable estimation of structural quantile effects. Econometrica 80 1533–1562.
  • [22] Ghosal, S. (1999). Asymptotic normality of posterior distributions in high-dimensional linear models. Bernoulli 5 315–331.
  • [23] Ghosal, S. (2000). Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity. J. Multivariate Anal. 74 49–68.
  • [24] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531.
  • [25] Ghosal, S. and van der Vaart, A. (2007). Convergence rates of posterior distributions for non-i.i.d. observations. Ann. Statist. 35 192–223.
  • [26] Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics. Springer, New York.
  • [27] Giné, E. and Nickl, R. (2011). Rates on contraction for posterior distributions in $L^{r}$-metrics, $1\leq r\leq\infty $. Ann. Statist. 39 2883–2911.
  • [28] Hall, P. and Horowitz, J. L. (2005). Nonparametric methods for inference in the presence of instrumental variables. Ann. Statist. 33 2904–2929.
  • [29] Hofinger, A. and Pikkarainen, H. K. (2007). Convergence rate for the Bayesian approach to linear inverse problems. Inverse Problems 23 2469–2484.
  • [30] Hofinger, A. and Pikkarainen, H. K. (2009). Convergence rates for linear inverse problems in the presence of an additive normal noise. Stoch. Anal. Appl. 27 240–257.
  • [31] Horowitz, J. L. (2011). Applied nonparametric instrumental variables estimation. Econometrica 79 347–394.
  • [32] Horowitz, J. L. (2012). Specification testing in nonparametric instrumental variable estimation. J. Econometrics 167 383–396.
  • [33] Horowitz, J. L. and Lee, S. (2007). Nonparametric instrumental variables estimation of a quantile regression model. Econometrica 75 1191–1208.
  • [34] Hu, Y. and Shiu, J. L. (2011). Nonparametric identification using instrumental variables: Sufficient conditions for completeness. Preprint.
  • [35] Jiang, W. and Tanner, M. A. (2008). Gibbs posterior for variable selection in high-dimensional classification and data mining. Ann. Statist. 36 2207–2231.
  • [36] Kato, K. (2013). Supplement to “Quasi-Bayesian analysis of nonparametric instrumental variables models”. DOI:10.1214/13-AOS1150SUPP.
  • [37] Kim, J.-Y. (2002). Limited information likelihood and Bayesian analysis. J. Econometrics 107 175–193. Information and entropy econometrics.
  • [38] Kleijn, B. J. K. and van der Vaart, A. W. (2006). Misspecification in infinite-dimensional Bayesian statistics. Ann. Statist. 34 837–877.
  • [39] Knapik, B. T., Szabó, B. T., van der Vaart, A. W. and van Zanten, J. H. (2012). Bayes procedures for adaptive inference in inverse problems for the white noise model. Available at arXiv:1209.3628.
  • [40] Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2011). Bayesian inverse problems with Gaussian priors. Ann. Statist. 39 2626–2657.
  • [41] Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2013). Bayesian recovery of the initial condition for the heat equation. Comm. Statist. Theory Methods 42 1294–1313.
  • [42] Kovchegov, Y. and Yildiz, N. (2012). Identification via completeness for discrete covariates and orthogonal polynomials. Preprint.
  • [43] Kress, R. (1999). Linear Integral Equations, 2nd ed. Applied Mathematical Sciences 82. Springer, New York.
  • [44] Liao, Y. and Jiang, W. (2011). Posterior consistency of nonparametric conditional moment restricted models. Ann. Statist. 39 3003–3031.
  • [45] Newey, W. K. and Powell, J. L. (2003). Instrumental variable estimation of nonparametric models. Econometrica 71 1565–1578.
  • [46] Santos, A. (2012). Inference in nonparametric instrumental variables with partial identification. Econometrica 80 213–275.
  • [47] Schennach, S. M. (2005). Bayesian exponentially tilted empirical likelihood. Biometrika 92 31–46.
  • [48] Scricciolo, C. (2006). Convergence rates for Bayesian density estimation of infinite-dimensional exponential families. Ann. Statist. 34 2897–2920.
  • [49] Severini, T. A. and Tripathi, G. (2006). Some identification issues in nonparametric linear models with endogenous regressors. Econometric Theory 22 258–278.
  • [50] Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687–714.
  • [51] Stuart, A. M. (2010). Inverse problems: A Bayesian perspective. Acta Numer. 19 451–559.
  • [52] Talagrand, M. (1996). New concentration inequalities in product spaces. Invent. Math. 126 505–563.
  • [53] Tian, L., Liu, J. S. and Wei, L. J. (2007). Implementation of estimating function-based inference procedures with Markov chain Monte Carlo samplers. J. Amer. Statist. Assoc. 102 881–888.
  • [54] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge.
  • [55] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York. With applications to statistics.
  • [56] Yosida, K. (1980). Functional Analysis, 6th ed. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 123. Springer, Berlin.
  • [57] Zhang, T. (2006). From $\epsilon $-entropy to KL-entropy: Analysis of minimum information complexity density estimation. Ann. Statist. 34 2180–2210.
  • [58] Zhang, T. (2006). Information-theoretic upper and lower bounds for statistical estimation. IEEE Trans. Inform. Theory 52 1307–1321.

Supplemental materials

  • Supplementary material: Supplement to “Quasi-Bayesian analysis of nonparametric instrumental variables models”. This supplemental file contains the additional technical proofs omitted in the main text, and some technical tools used in the proofs.