The Annals of Statistics

Approximating conditional distribution functions using dimension reduction

Peter Hall and Qiwei Yao

Full-text: Open access


Motivated by applications to prediction and forecasting, we suggest methods for approximating the conditional distribution function of a random variable Y given a dependent random d-vector X. The idea is to estimate not the distribution of Y|X, but that of YTX, where the unit vector θ is selected so that the approximation is optimal under a least-squares criterion. We show that θ may be estimated root-n consistently. Furthermore, estimation of the conditional distribution function of Y, given θTX, has the same first-order asymptotic properties that it would enjoy if θ were known. The proposed method is illustrated using both simulated and real-data examples, showing its effectiveness for both independent datasets and data from time series. Numerical work corroborates the theoretical result that θ can be estimated particularly accurately.

Article information

Ann. Statist., Volume 33, Number 3 (2005), 1404-1421.

First available in Project Euclid: 1 July 2005

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62E17: Approximations to distributions (nonasymptotic)
Secondary: 62G05: Estimation 62G20: Asymptotic properties

Conditional distribution cross-validation dimension reduction kernel methods leave-one-out method local linear regression nonparametric regression prediction root-n consistency time series analysis


Hall, Peter; Yao, Qiwei. Approximating conditional distribution functions using dimension reduction. Ann. Statist. 33 (2005), no. 3, 1404--1421. doi:10.1214/009053604000001282.

Export citation


  • Adali, T., Liu, X. and Sonmez, M. K. (1997). Conditional distribution learning with neural networks and its application to channel equalization. IEEE Trans. Signal Processing \bf45 1051–1064.
  • Bashtannyk, D. M. and Hyndman, R. J. (2001). Bandwidth selection for kernel conditional density estimation. Comput. Statist. Data Anal. \bf36 279–298.
  • Bhattacharya, P. K. and Gangopadhyay, A. K. (1990). Kernel and nearest-neighbor estimation of a conditional quantile. Ann. Statist. \bf18 1400–1415.
  • Bond, S. A. and Patel, K. (2000). The conditional distribution of real estate returns: Relating time variation in higher moments to downside risk measurement. Technical report, Dept. Land Economy, Univ. Cambridge.
  • Cai, Z. (2002). Regression quantiles for time series. Econometric Theory \bf18 169–192.
  • Fan, J., Heckman, N. E. and Wand, M. P. (1995). Local polynomial kernel regression for generalized linear models and quasi-likelihood functions. J. Amer. Statist. Assoc. \bf90 141–150.
  • Fan, J. and Yao, Q. (2003). Nonlinear Time Series: Nonparametric and Parametric Methods. Springer, New York.
  • Fan, J., Yao, Q. and Tong, H. (1996). Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika \bf83 189–206.
  • Foresi, S. and Paracchi, F. (1992). The conditional distribution of excess returns: An empirical analysis. Working Paper 92-49, C. V. Starr Center, New York Univ.
  • Friedman, J. H. (1987). Exploratory projection pursuit. J. Amer. Statist. Assoc. \bf82 249–266.
  • Friedman, J. H. and Stuetzle, W. (1981). Projection pursuit regression. J. Amer. Statist. Assoc. \bf76 817–823.
  • Friedman, J. H., Stuetzle, W. and Schroeder, A. (1984). Projection pursuit density estimation. J. Amer. Statist. Assoc. \bf79 599–608.
  • Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Application. Academic Press, New York.
  • Hall, P., Wolff, R. C. L. and Yao, Q. (1999). Methods for estimating a conditional distribution function. J. Amer. Statist. Assoc. \bf94 154–163.
  • Hall, P. and Yao, Q. (2002). Estimating conditional distribution functions using dimension reduction. Research Report 87, Dept. Statistics, London School of Economics. Available at
  • Härdle, W., Hall, P. and Ichimura, H. (1993). Optimal smoothing in single-index models. Ann. Statist. \bf21 157–178.
  • Huber, P. J. (1985). Projection pursuit (with discussion). Ann. Statist. \bf13 435–525.
  • Hyndman, R. J., Bashtannyk, D. M. and Grunwald, G. K. (1996). Estimating and visualizing conditional densities. J. Comput. Graph. Statist. \bf5 315–336.
  • Hyndman, R. J. and Yao, Q. (2002). Nonparametric estimation and symmetry tests for conditional density functions. J. Nonparametr. Statist. \bf14 259–278.
  • Ichimura, H. (1993). Semiparametric least squares (SLS) and weighted SLS estimation of single-index models. J. Econometrics \bf58 71–120.
  • Jones, M. C. and Sibson, R. (1987). What is projection pursuit? (with discussion). J. Roy. Statist. Soc. Ser. A \bf150 1–36.
  • Klein, R. and Spady, R. (1993). An efficient semiparametric estimator for binary response models. Econometrica \bf61 387–422.
  • Li, K.-C. (1991). Sliced inverse regression for dimension reduction (with discussion). J. Amer. Statist. Assoc. \bf86 316–342.
  • Moran, P. A. P. (1953). The statistical analysis of the Canadian lynx cycle. I. Structure and prediction. Australian J. Zoology \bf1 163–173.
  • Posse, C. (1995). Projection pursuit exploratory data analysis. Comput. Statist. Data Anal. \bf20 669–687.
  • Powell, J. L., Stock, J. H. and Stoker, T. M. (1989). Semiparametric estimation of index coefficients. Econometrica \bf57 1403–1430.
  • Press, W. H., Teukolsky, S. A., Vetterling, W. T. and Flannery, B. P. (1992). Numerical Recipes in C. The Art of Scientific Computing, 2nd ed. Cambridge Univ. Press.
  • Rosenblatt, M. (1969). Conditional probability density and regression estimators. In Multivariate Analysis II (P. Krishnaiah, ed.) 25–31. Academic Press, New York.
  • Sheather, S. J. and Marron, J. S. (1990). Kernel quantile estimators. J. Amer. Statist. Assoc. \bf85 410–416.
  • Tiao, G. C. and Tsay, R. S. (1994). Some advances in nonlinear and adaptive modeling in time series. J. Forecasting \bf13 109–131.
  • Watanabe, T. (2000). Excess kurtosis of conditional distribution for daily stock returns: The case of Japan. Applied Economics Letters \bf7 353–355.
  • Yin, X. and Cook, R. D. (2002). Dimension reduction for the conditional $k$th moment in regression. J. R. Stat. Soc. Ser. B Stat. Methodol. \bf64 159–175.
  • Yu, K. and Jones, M. C. (1998). Local linear quantile regression. J. Amer. Statist. Assoc. \bf93 228–237.