Open Access
October 2009 Conditional predictive inference post model selection
Hannes Leeb
Ann. Statist. 37(5B): 2838-2876 (October 2009). DOI: 10.1214/08-AOS660


We give a finite-sample analysis of predictive inference procedures after model selection in regression with random design. The analysis is focused on a statistically challenging scenario where the number of potentially important explanatory variables can be infinite, where no regularity conditions are imposed on unknown parameters, where the number of explanatory variables in a “good” model can be of the same order as sample size and where the number of candidate models can be of larger order than sample size. The performance of inference procedures is evaluated conditional on the training sample. Under weak conditions on only the number of candidate models and on their complexity, and uniformly over all data-generating processes under consideration, we show that a certain prediction interval is approximately valid and short with high probability in finite samples, in the sense that its actual coverage probability is close to the nominal one and in the sense that its length is close to the length of an infeasible interval that is constructed by actually knowing the “best” candidate model. Similar results are shown to hold for predictive inference procedures other than prediction intervals like, for example, tests of whether a future response will lie above or below a given threshold.


Download Citation

Hannes Leeb. "Conditional predictive inference post model selection." Ann. Statist. 37 (5B) 2838 - 2876, October 2009.


Published: October 2009
First available in Project Euclid: 17 July 2009

zbMATH: 1173.62026
MathSciNet: MR2541449
Digital Object Identifier: 10.1214/08-AOS660

Primary: 62G15
Secondary: 62H12 , 62J05 , 62J07

Keywords: approximately honest and short prediction interval , conditional coverage probability , finite sample analysis , Predictive inference post model selection , regression with random design

Rights: Copyright © 2009 Institute of Mathematical Statistics

Vol.37 • No. 5B • October 2009
Back to Top