Bernoulli

  • Bernoulli
  • Volume 4, Number 2 (1998), 203-272.

On large-deviation efficiency in statistical inference

Anatolii Puhalskii and Vladimir Spokoiny

Full-text: Open access

Abstract

We present a general approach to statistical problems with criteria based on probabilities of large deviations. Our main idea, which originates from similarity in the definitions of the large-deviation principle (LDP) and weak convergence, is to develop a large-deviation analogue of asymptotic decision theory. We introduce the concept of the LPD for sequences of statistical experiments, which parallels the concept of weak convergence of experiments, and prove that, in analogy with Le Cam's minimax theorem, the LPD provides an asymptotic lower bound for the sequence of appropriately defined minimax risks. We also show that the bound is tight and give a method of constructing decisions whose asymptotic risk is arbitrarily close to the bound. The construction is further specified for hypothesis testing and estimation problems.

Article information

Source
Bernoulli, Volume 4, Number 2 (1998), 203-272.

Dates
First available in Project Euclid: 26 March 2007

Permanent link to this document
https://projecteuclid.org/euclid.bj/1174937295

Mathematical Reviews number (MathSciNet)
MR1632979

Zentralblatt MATH identifier
0954.62006

Keywords
Bahadur efficiency Chernoff's function large-deviation large-deviation principle minimax risk statistical experiments

Citation

Puhalskii, Anatolii; Spokoiny, Vladimir. On large-deviation efficiency in statistical inference. Bernoulli 4 (1998), no. 2, 203--272. https://projecteuclid.org/euclid.bj/1174937295


Export citation

References

  • [1] Aubin, J.-P. (1984) L'Analyse non Linéaire et ses Motivations Économiques. Paris: Masson.
  • [2] Aubin, J.-P. and Ekeland, I. (1984) Applied Nonlinear Analysis. New York: Wiley.
  • [3] Bahadur, R. (1960) On the asymptotic efficiency of tests and estimators. Sankhya, 22, 229-252.
  • [4] Bahadur, R., Zabell, S., and Gupta, J. (1980) Large deviations, tests, and estimates. In I.M. Chaterabarli (ed.), Asymptotic Theory of Statistical Tests and Estimation, pp. 33-64. New York: Academic Press.
  • [5] Basu, D. (1956) On the concept of asymptotic efficiency. Sankhya, 17, 193-196.
  • [6] Billingsley, P. (1968) Convergence of Probability Measures. New York: Wiley.
  • [7] Borovkov, A. and Mogulskii, A. (1992a) Large deviations and statistical invariance principle. Theory Probab. Appl., 37, 11-18.
  • [8] Borovkov, A. and Mogulskii, A. (1992b) Large deviations and testing of statistical hypotheses. Proc. Inst. Math. Russian Acad. Sci., Siberian Division, 19. English transl.: Siberian Adv. Math., 2(3, 4), 1992; 3(1, 2), 1993.
  • [9] Bryc, W. (1990) Large deviations by the asymptotic value method. In M.A. Pinsky (ed.), Diffusion Processes and Related Problems in Analysis, Vol. 1, pp. 447-472. Birkhäuser.
  • [10] Chaganty, N. (1993) Large deviations for joint distributions and statistical applications. Technical Report TR93-2, Department of Mathematics and Statistics, Old Dominion University, Norfolk, VA.
  • [11] Chernoff, H. (1952) A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Ann. Math. Statist., 23, 497-507.
  • [12] Dawson, D. and Gärtner, J. (1987) Large deviations from the McKean-Vlasov limit for weakly interacting diffusions. Stochastics Stochastics Rep., 20, 247-308.
  • [13] Dembo, A. and Zajic, T. (1995) Large deviations: from empirical mean and measure to partial sums processes. Stochastic Process Appl., 57, 191-224.
  • [14] Deuschel, J. and Stroock, D. (1989) Large Deviations. Boston: Academic Press.
  • [15] Engelking, R. (1977) General Topology. Warszawa: PWN.
  • [16] Ermakov, M. (1993) Large deviations of empirical measures and hypothesis testing. Zap. Nauchn. Sem. LOMI RAN, 207, 37-60.
  • [17] Freidlin, M. and Wentzell, D. (1979) Random Perturbations of Dynamical Systems. Moscow: Nauka [in Russian]. English translation: Springer (1984).
  • [18] Fu, J. (1982) Large sample point estimation: a large deviation theory approach. Ann. Statist., 10, 762-771.
  • [19] Ibragimov, I. and Khasminskii, R. (1981) Statistical Estimation: Asymptotic Theory. New York: Springer.
  • [20] Ibragimov, I. and Radavicius, M. (1981) Probability of large deviations for the maximum likelihood estimator. Soviet Math. Dokl., 23(2), 403-406.
  • [21] Kallenberg, W. (1981) Bahadur deficiency of likelihood ratio tests in exponential families. J. Multivariate Anal., 11, 506-531.
  • [22] Kallenberg, W. (1983) Intermediate efficiency, theory and examples. Ann. Statist., 11, 170-182.
  • [23] Kelley, J. (1955) General Topology. New York: Van Nostrand.
  • [24] Korostelev, A. (1991) A minimaxity criterion in nonparametric regression based on large deviations probabilities. Ann. Statist., 24, 1075-1083.
  • [25] Korostelev, A. (1995) Minimax large deviations risk in change-point problems. Stochastic Process. Appl. (To appear).
  • [26] Korostelev, A. and Leonov, S. (1995) Minimax Bahadur efficiency for small confidence intervals. Discussion paper 37, Sonderforschungsbereich 373, Humboldt University, Berlin.
  • [27] Korostelev, A. and Spokoiny, V. (1996) Exact asymptotics of minimax Bahadur risk in Lipschitz regression. Statistics., 28, 13-24.
  • [28] Krasnoselskii, M. and Rutickii, Y. (1961) Convex Functions and Orlicz Spaces. Groningen: Noordhoff.
  • [29] Kullback, S. (1959) Information Theory and Statistics. New York: Wiley.
  • [30] Le Cam, L. (1960) Locally asymptotically normal families of distributions. Univ. California Publ. Statist., 3, 27-98.
  • [31] Le Cam, L. (1972) Limits of experiments. In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, pp. 245-261. Berkeley: University of California Press.
  • [32] Le Cam, L. (1986) Asymptotic Methods in Statistical Decision Theory. New York: Springer-Verlag.
  • [33] Lynch, J. and Sethuraman, J. (1987) Large deviations for processes with independent increments. Ann. Probab., 15(2), 610-627.
  • [34] Mogulskii, A.A. (1976) Large deviations for trajectories of multidimensional random walks. Theory Probab. Appl., 21(2), 300-315.
  • [35] Puhalskii, A. (1991) On functional principle of large deviations. In V. Sazonov and T. Shervashidze (eds), New Trends in Probability and Statistics, Vol. 1, pp. 198-218. Utrecht: VSP/Moks'las.
  • [36] Puhalskii, a. (1993) On the theory of large deviations. Theory Probab. Appl., 38(3), 490-497.
  • [37] Puhalskii, A. (1994a) Large deviations of semimartingales via convergence of the predictable characteristics. Stochastics Stochastics Rep., 49, 27-85.
  • [38] Puhalskii, A. (1994b) The method of stochastic exponentials for large deviations. Stochastic Process Appl., 54, 45-70.
  • [39] Puhalskii, A. (1995) Large deviation analysis of the single server queue. Queueing Systems Theory Appl., 21, 5-66.
  • [40] Puhalskii, A. (1997) Large deviations of semimartingales: a maxingale problem approach. I. Limits as solutions to a maxingale problem. (To appear in Stoch. Stoch. Rep.)
  • [41] Puhalskii, A. (1996) Large deviations of the statistical empirical process. In A.N. Shiryaev et al. (eds), Frontiers in Pure and Applied Probability II, pp. 163-170. Moscow: TVP.
  • [42] Radavicius, M. (1983) On the probability of large deviations of maximum likelihood estimators. Soviet Math. Dokl., 27(1), 127-131.
  • [43] Radavicius, M. (1991) From asymptotic efficiency in minimax sense to Bahadur efficiency. In V. Sazonov and T. Shervashidze (eds), New Trends in Probability and Statistics, Vol. 1, pp. 629-635. Utrecht: VSP/Moks'las.
  • [44] Rao, C. (1963) Criteria of estimation in large samples. Sankhya A, 25, 189-206.
  • [45] Rockafellar, R. (1970) Convex Analysis. Princeton, NJ: Princeton University Press.
  • [46] Rubin, H. and Rukhin, A. (1983) Convergence rates of large deviations probabilities for point estimators. Statist. Probab. Lett., 1, 197-202.
  • [47] Sanov, I. (1957) On the probability of large deviations of random variables. Mat. Sb., 42 [in Russian]. English transl.: Sel. Transl. Math. Statist. Probab., 1, 213-244 (1961).
  • [48] Schwartz, L. (1973) Radon Measures on Arbitrary Topological Spaces and Cylindrical Measures. Oxford: Oxford University Press.
  • [49] Shiryaev, A. and Spokoiny, V. (1997) Statistical Experiments and Decisions: Asymptotic Theory. Springer. Forthcoming.
  • [50] Sievers, G. (1978) Estimates of location: a large deviations comparison. Ann. Statist., 6, 610-618.
  • [51] Strasser, H. (1985) Mathematical Theory of Statistics: Statistical Experiments and Asymptotic Decision Theory. Berlin: de Gruyter.
  • [52] Varadhan, S. (1966) Asymptotic probabilities and differential equations. Comm. Pure Appl. Math., 19(3), 261-286.
  • [53] Varadhan, S. (1984) Large Deviations and Applications. Philadelphia: SIAM.
  • [54] Vervaat, W. (1988) Narrow and vague convergence of set functions. Statist. Probab. Lett., 6(5), 295-298.
  • [55] Wieand, H. (1976) A condition under which the Pitman and Bahadur approaches to efficiency coincide. Ann. Statist., 4, 1003-1011.