Open Access
VOL. 9 | 2013 Multivariate regression through affinely weighted penalized least squares
Rudolf Beran

Editor(s) M. Banerjee, F. Bunea, J. Huang, V. Koltchinskii, M. H. Maathuis

Inst. Math. Stat. (IMS) Collect., 2013: 33-46 (2013) DOI: 10.1214/12-IMSCOLL904


Stein’s [In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability 1 (1956) 197–206 University of California Press] asymptotically superior shrinkage estimator of $p$ univariate means may be rederived through adaptive penalized least squares (PLS) estimation in a regression model with one nominal covariate. This paper treats adaptive PLS estimators for $p$ unknown $d$-dimensional mean vectors, each of which depends on $k_{0}$ scalar covariates that may be nominal or ordinal. The initial focus is on complete regression designs, not necessarily balanced: for every $k_{0}$-tuple of possible covariate values, at least one observation is made on the corresponding mean vector. The results include definition of suitable candidate classes of PLS estimators in multivariate regression problems, comparison of these candidate estimators through their estimated quadratic risks under a general model that makes no assumptions about the regression function, and supporting asymptotic theory as the number $p$ of covariate-value combinations observed tends to infinity. Empirical process theory establishes that: (a) estimated risks converge to their intended targets uniformly over large classes of candidate PLS estimators; (b) a candidate PLS estimator that minimizes estimated risk within such a class has risk that converges to the minimal possible risk within the class. Extension of the results to incomplete regression designs is outlined. The Efron-Morris [ Journal of the American Statistical Association 68 (1973) 117–130] and Beran [ Annals of the Institute of Statistical Mathematics 60 (2008) 843–864] estimators for multivariate means in balanced complete designs are seen as special cases.


Published: 1 January 2013
First available in Project Euclid: 8 March 2013

zbMATH: 1327.62419
MathSciNet: MR3186747

Digital Object Identifier: 10.1214/12-IMSCOLL904

Primary: 62F12 , 62J07
Secondary: 62H12

Keywords: Adaptation , multiple penalties , quadratic risk , unbalanced design

Rights: Copyright © 2010, Institute of Mathematical Statistics

Back to Top