The Annals of Statistics

Contour regression: A general approach to dimension reduction

Bing Li, Hongyuan Zha, and Francesca Chiaromonte

Full-text: Open access

Enhanced PDF (327 KB)

Abstract

We propose a novel approach to sufficient dimension reduction in regression, based on estimating contour directions of small variation in the response. These directions span the orthogonal complement of the minimal space relevant for the regression and can be extracted according to two measures of variation in the response, leading to simple and general contour regression (SCR and GCR) methodology. In comparison with existing sufficient dimension reduction techniques, this contour-based methodology guarantees exhaustive estimation of the central subspace under ellipticity of the predictor distribution and mild additional assumptions, while maintaining $\sqrt{n}$-consistency and computational ease. Moreover, it proves robust to departures from ellipticity. We establish population properties for both SCR and GCR, and asymptotic properties for SCR. Simulations to compare performance with that of standard techniques such as ordinary least squares, sliced inverse regression, principal Hessian directions and sliced average variance estimation confirm the advantages anticipated by the theoretical analyses. We demonstrate the use of contour-based methods on a data set concerning soil evaporation.

Article information

Source
Ann. Statist., Volume 33, Number 4 (2005), 1580-1616.

Dates
First available in Project Euclid: 5 August 2005

Permanent link to this document
https://projecteuclid.org/euclid.aos/1123250223

Digital Object Identifier
doi:10.1214/009053605000000192

Mathematical Reviews number (MathSciNet)
MR2166556

Zentralblatt MATH identifier
1078.62033

Subjects
Primary: 62G08: Nonparametric regression
Secondary: 62G09: Resampling methods 62H05: Characterization and structure theory

Keywords
Central subspace empirical directions PCA nonparametric regression data visualization

Citation

Li, Bing; Zha, Hongyuan; Chiaromonte, Francesca. Contour regression: A general approach to dimension reduction. Ann. Statist. 33 (2005), no. 4, 1580--1616. doi:10.1214/009053605000000192. https://projecteuclid.org/euclid.aos/1123250223


Export citation

References

  • Chiaromonte, F. and Cook, R. D. (2002). Sufficient dimension reduction and graphics in regression. Ann. Inst. Statist. Math. 54 768–795.
    Zentralblatt MATH: 1047.62066
    Digital Object Identifier: doi:10.1023/A:1022411301790
  • Chiaromonte, F., Cook, R. D. and Li, B. (2002). Sufficient dimension reduction in regressions with categorical predictors. Ann. Statist. 30 475–497.
    Zentralblatt MATH: 1012.62036
    Digital Object Identifier: doi:10.1214/aos/1021379862
    Project Euclid: euclid.aos/1021379862
  • Cook, R. D. (1994). Using dimension reduction subspaces to identify important inputs in models of physical systems. In Proc. Section on Physical and Engineering Sciences 18–25. Amer. Statist. Assoc., Alexandria, VA.
  • Cook, R. D. (1998). Regression Graphics. Wiley, New York.
    Zentralblatt MATH: 0903.62001
  • Cook, R. D. and Li, B. (2002). Dimension reduction for conditional mean in regression. Ann. Statist. 30 455–474.
    Zentralblatt MATH: 1012.62035
    Digital Object Identifier: doi:10.1214/aos/1021379861
    Project Euclid: euclid.aos/1021379861
  • Cook, R. D. and Nachtsheim, C. J. (1994). Reweighting to achieve elliptically contoured covariates in regression. J. Amer. Statist. Assoc. 89 592–599.
    Zentralblatt MATH: 0799.62078
    Digital Object Identifier: doi:10.1080/01621459.1994.10476784
  • Cook, R. D. and Weisberg, S. (1991). Discussion of “Sliced inverse regression for dimension reduction,” by K.-C. Li. J. Amer. Statist. Assoc. 86 328–332.
  • Dawid, A. P. (1979). Conditional independence in statistical theory (with discussion). J. Roy. Statist. Soc. Ser. B 41 1–31.
    Mathematical Reviews (MathSciNet): MR535541
    Zentralblatt MATH: 0408.62004
    Digital Object Identifier: doi:10.1111/j.2517-6161.1979.tb01052.x
  • Duan, N. and Li, K.-C. (1991). Slicing regression: A link-free regression method. Ann. Statist. 19 505–530.
    Zentralblatt MATH: 0738.62070
    Digital Object Identifier: doi:10.1214/aos/1176348109
    Project Euclid: euclid.aos/1176348109
  • Eaton, M. L. and Tyler, D. (1994). The asymptotic distribution of singular values with application to canonical correlations and correspondence analysis. J. Multivariate Anal. 50 238–264.
    Zentralblatt MATH: 0805.62020
    Digital Object Identifier: doi:10.1006/jmva.1994.1041
  • Fan, J. (1993). Local linear regression smoothers and their minimax efficiencies. Ann. Statist. 21 196–216.
    Zentralblatt MATH: 0773.62029
    Digital Object Identifier: doi:10.1214/aos/1176349022
    Project Euclid: euclid.aos/1176349022
  • Freund, R. J. (1979). Multicollinearity etc., some “new” examples. ASA Proc. Statistical Computing Section 111–112. Amer. Statist. Assoc., Washington.
  • Hristache, M., Juditsky, A., Polzehl, J. and Spokoiny, V. (2001). Structure adaptive approach for dimension reduction. Ann. Statist. 29 1537–1566.
    Zentralblatt MATH: 1043.62052
    Project Euclid: euclid.aos/1015345954
  • Li, B., Cook, R. D. and Chiaromonte, F. (2003). Dimension reduction for the conditional mean in regressions with categorical predictors. Ann. Statist. 31 1636–1668.
    Zentralblatt MATH: 1042.62037
    Digital Object Identifier: doi:10.1214/aos/1065705121
    Project Euclid: euclid.aos/1065705121
  • Li, K.-C. (1991). Sliced inverse regression for dimension reduction (with discussion). J. Amer. Statist. Assoc. 86 316–342.
  • Li, K.-C. (1992). On principal Hessian directions for data visualization and dimension reduction: Another application of Stein's lemma. J. Amer. Statist. Assoc. 87 1025–1039.
    Zentralblatt MATH: 0765.62003
    Digital Object Identifier: doi:10.1080/01621459.1992.10476258
  • Li, K.-C. and Duan, N. (1989). Regression analysis under link violation. Ann. Statist. 17 1009–1052.
    Zentralblatt MATH: 0753.62041
    Digital Object Identifier: doi:10.1214/aos/1176347254
    Project Euclid: euclid.aos/1176347254
  • Nadaraya, E. A. (1964). On estimating regression. Theory Probab. Appl. 9 141–142.
    Zentralblatt MATH: 0136.40902
  • Peters, B. C., Jr., Redner, R. and Decell, H. P., Jr. (1978). Characterizations of linear sufficient statistics. Sankhyā Ser. A 40 303–309.
  • Serfling, R. (1980). Approximation Theorems of Mathematical Statistics. Wiley, New York.
    Zentralblatt MATH: 0538.62002
  • Stone, C. J. (1977). Consistent nonparametric regression (with discussion). Ann. Statist. 5 595–645.
    Zentralblatt MATH: 0366.62051
    Digital Object Identifier: doi:10.1214/aos/1176343886
    Project Euclid: euclid.aos/1176343886
  • Watson, G. S. (1964). Smooth regression analysis. Sankhyā Ser. A 26 359–372.
  • Xia, Y., Tong, H., Li, W. K. and Zhu, L.-X. (2002). An adaptive estimation of dimension reduction space. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 363–410.
    Zentralblatt MATH: 1091.62028
    Digital Object Identifier: doi:10.1111/1467-9868.03411

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more