Statistical Science
- Statist. Sci.
- Volume 26, Number 3 (2011), 369-387.
Covariance Estimation: The GLM and Regularization Perspectives
Full-text: Open access
Abstract
Finding an unconstrained and statistically interpretable reparameterization of a covariance matrix is still an open problem in statistics. Its solution is of central importance in covariance estimation, particularly in the recent high-dimensional data environment where enforcing the positive-definiteness constraint could be computationally expensive. We provide a survey of the progress made in modeling covariance matrices from two relatively complementary perspectives: (1) generalized linear models (GLM) or parsimony and use of covariates in low dimensions, and (2) regularization or sparsity for high-dimensional data. An emerging, unifying and powerful trend in both perspectives is that of reducing a covariance estimation problem to that of estimating a sequence of regression problems. We point out several instances of the regression-based formulation. A notable case is in sparse estimation of a precision matrix or a Gaussian graphical model leading to the fast graphical LASSO algorithm. Some advantages and limitations of the regression-based Cholesky decomposition relative to the classical spectral (eigenvalue) and variance-correlation decompositions are highlighted. The former provides an unconstrained and statistically interpretable reparameterization, and guarantees the positive-definiteness of the estimated covariance matrix. It reduces the unintuitive task of covariance estimation to that of modeling a sequence of regressions at the cost of imposing an a priori order among the variables. Elementwise regularization of the sample covariance matrix such as banding, tapering and thresholding has desirable asymptotic properties and the sparse estimated covariance matrix is positive definite with probability tending to one for large samples and dimensions.
Article information
Source
Statist. Sci., Volume 26, Number 3 (2011), 369-387.
Dates
First available in Project Euclid: 31 October 2011
Permanent link to this document
https://projecteuclid.org/euclid.ss/1320066926
Digital Object Identifier
doi:10.1214/11-STS358
Mathematical Reviews number (MathSciNet)
MR2917961
Zentralblatt MATH identifier
1246.62139
Keywords
Bayesian estimation Cholesky decomposition dependence and correlation graphical models longitudinal data parsimony penalized likelihood precision matrix sparsity spectral decomposition variance-correlation decomposition
Citation
Pourahmadi, Mohsen. Covariance Estimation: The GLM and Regularization Perspectives. Statist. Sci. 26 (2011), no. 3, 369--387. doi:10.1214/11-STS358. https://projecteuclid.org/euclid.ss/1320066926
References
- Anderson, T. W. (1973). Asymptotically efficient estimation of covariance matrices with linear structure. Ann. Statist. 1 135–141.Mathematical Reviews (MathSciNet): MR331612
Zentralblatt MATH: 0296.62022
Digital Object Identifier: doi:10.1214/aos/1193342389
Project Euclid: euclid.aos/1193342389 - Anderson, T. W. (2003). An Introduction to Multivariate Statistical Analysis, 3rd ed. Wiley, Hoboken, NJ.Mathematical Reviews (MathSciNet): MR1990662
- Banerjee, O., El Ghaoui, L. and d’Aspremont, A. (2008). Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Res. 9 485–516.Mathematical Reviews (MathSciNet): MR2417243
- Barnard, J., McCulloch, R. and Meng, X.-L. (2000). Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage. Statist. Sinica 10 1281–1311.
- Bartlett, M. S. (1933). On the theory of statistical regression. Proc. Roy. Soc. Edinburgh 53 260–283.
- Bickel, P. J. and Levina, E. (2004). Some theory of Fisher’s linear discriminant function, ‘naive Bayes,’ and some alternatives when there are many more variables than observations. Bernoulli 10 989–1010.Mathematical Reviews (MathSciNet): MR2108040
Digital Object Identifier: doi:10.3150/bj/1106314847
Project Euclid: euclid.bj/1106314847 - Bickel, P. J. and Levina, E. (2008a). Regularized estimation of large covariance matrices. Ann. Statist. 36 199–227.Mathematical Reviews (MathSciNet): MR2387969
Zentralblatt MATH: 1132.62040
Digital Object Identifier: doi:10.1214/009053607000000758
Project Euclid: euclid.aos/1201877299 - Bickel, P. J. and Levina, E. (2008b). Covariance regularization by thresholding. Ann. Statist. 36 2577–2604.Mathematical Reviews (MathSciNet): MR2485008
Zentralblatt MATH: 1196.62062
Digital Object Identifier: doi:10.1214/08-AOS600
Project Euclid: euclid.aos/1231165180 - Bilmes, J. A. (2000). Factored sparse inverse covariance matrices. In IEEE International Conference on Accoustics, Speech and Signal Processing (Istanbul, Turkey) 2 II1009–II1012.
- Boik, R. J. (2002). Spectral models for covariance matrices. Biometrika 89 159–182.Mathematical Reviews (MathSciNet): MR1888370
Zentralblatt MATH: 0997.62046
Digital Object Identifier: doi:10.1093/biomet/89.1.159 - Bondell, H. D., Krishna, A. and Ghosh, S. K. (2010). Joint variable selection for fixed and random effects in linear mixed-effects models. Biometrics 66 1069–1077.
- Box, G. E. P., Jenkins, G. M. and Reinsel, G. C. (1994). Time Series Analysis: Forecasting and Control, 3rd ed. Prentice Hall, Englewood Cliffs, NJ.Mathematical Reviews (MathSciNet): MR1312604
- Brown, P. J., Le, N. D. and Zidek, J. V. (1994). Inference for a covariance matrix. In Aspects of Uncertainty (P. R. Freeman and A. F. M. Smith, eds.) 77–92. Wiley, Chichester.
- Cai, T. T., Zhang, C.-H. and Zhou, H. H. (2010). Optimal rates of convergence for covariance matrix estimation. Ann. Statist. 38 2118–2144.Mathematical Reviews (MathSciNet): MR2676885
Zentralblatt MATH: 1202.62073
Digital Object Identifier: doi:10.1214/09-AOS752
Project Euclid: euclid.aos/1278861244 - Cannon, M. J., Warner, L., Taddei, J. A. and Kleinbaum, D. G. (2001). What can go wrong when you assume that correlated data are independent: An illustration from the evaluation of a childhood health intervention in Brazil. Statist. Med. 20 1461–1467.
- Carroll, R. J. (2003). Variances are not always nuisance parameters. Biometrics 59 211–220.Mathematical Reviews (MathSciNet): MR1987387
Digital Object Identifier: doi:10.1111/1541-0420.t01-1-00027 - Carroll, R. J. and Ruppert, D. (1988). Transformation and Weighting in Regression. Chapman & Hall, New York.
- Chang, C. and Tsay, R. S. (2010). Estimation of covariance matrix via the sparse Cholesky factor with lasso. J. Statist. Plann. Inference 140 3858–3873.Mathematical Reviews (MathSciNet): MR2674171
Zentralblatt MATH: 05788991
Digital Object Identifier: doi:10.1016/j.jspi.2010.04.048 - Chen, Z. and Dunson, D. B. (2003). Random effects selection in linear mixed models. Biometrics 59 762–769.Mathematical Reviews (MathSciNet): MR2025100
Digital Object Identifier: doi:10.1111/j.0006-341X.2003.00089.x - Chiu, T. Y. M., Leonard, T. and Tsui, K.-W. (1996). The matrix-logarithmic covariance model. J. Amer. Statist. Assoc. 91 198–210.Mathematical Reviews (MathSciNet): MR1394074
Zentralblatt MATH: 0870.62043
Digital Object Identifier: doi:10.2307/2291396 - Cressie, N. A. C. (1993). Statistics for Spatial Data, rev ed. Wiley, New York.
- Daniels, M. J. (2005). A class of shrinkage priors for the dependence structure in longitudinal data. J. Statist. Plann. Inference 127 119–130.Mathematical Reviews (MathSciNet): MR2103028
Zentralblatt MATH: 1054.62019
Digital Object Identifier: doi:10.1016/j.jspi.2003.09.026 - Daniels, M. J. and Hogan, J. W. (2008). Missing Data in Longitudinal Studies: Strategies for Bayesian Modeling and Sensitivity Analysis. Monographs on Statistics and Applied Probability 109. Chapman & Hall/CRC, Boca Raton, FL.Mathematical Reviews (MathSciNet): MR2459796
- Daniels, M. J. and Kass, R. E. (1999). Nonconjugate Bayesian estimation of covariance matrices and its use in hierarchical models. J. Amer. Statist. Assoc. 94 1254–1263.Mathematical Reviews (MathSciNet): MR1731487
Zentralblatt MATH: 1069.62508
Digital Object Identifier: doi:10.2307/2669939 - Daniels, M. J. and Kass, R. E. (2001). Shrinkage estimators for covariance matrices. Biometrics 57 1173–1184.Mathematical Reviews (MathSciNet): MR1950425
Digital Object Identifier: doi:10.1111/j.0006-341X.2001.01173.x - Daniels, M. J. and Pourahmadi, M. (2002). Bayesian analysis of covariance matrices and dynamic models for longitudinal data. Biometrika 89 553–566.Mathematical Reviews (MathSciNet): MR1929162
Zentralblatt MATH: 1036.62019
Digital Object Identifier: doi:10.1093/biomet/89.3.553 - Daniels, M. J. and Pourahmadi, M. (2009). Modeling covariance matrices via partial autocorrelations. J. Multivariate Anal. 100 2352–2363.Mathematical Reviews (MathSciNet): MR2560376
Zentralblatt MATH: 1175.62090
Digital Object Identifier: doi:10.1016/j.jmva.2009.04.015 - Dégerine, S. and Lambert-Lacroix, S. (2003). Partial autocorrelation function of a nonstationary time series J. Multivariate Anal. 89 135–147.
- Dempster, A. (1972). Covariance selection models. Biometrics 28 157–175.
- Dey, D. K. and Srinivasan, C. (1985). Estimation of a covariance matrix under Stein’s loss. Ann. Statist. 13 1581–1591.Mathematical Reviews (MathSciNet): MR811511
Zentralblatt MATH: 0582.62042
Digital Object Identifier: doi:10.1214/aos/1176349756
Project Euclid: euclid.aos/1176349756 - Diggle, P., Liang, K. Y., Zeger, S. L. and Heagerty, P. J. (2002). Analysis of Longitudinal Data, 2nd ed. Clarendon Press, Oxford.
- Eaves, D. and Chang, T. (1992). Priors for ordered conditional variance and vector partial correlation. J. Multivariate Anal. 41 43–55.Mathematical Reviews (MathSciNet): MR1156680
Zentralblatt MATH: 0745.62024
Digital Object Identifier: doi:10.1016/0047-259X(92)90056-L - Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression (with discussion). Ann. Statist. 32 407–499.Mathematical Reviews (MathSciNet): MR2060166
Zentralblatt MATH: 1091.62054
Digital Object Identifier: doi:10.1214/009053604000000067
Project Euclid: euclid.aos/1083178935 - El Karoui, N. (2008a). Operator norm consistent estimation of large-dimensional sparse covariance matrices. Ann. Statist. 36 2717–2756.Mathematical Reviews (MathSciNet): MR2485011
Zentralblatt MATH: 1196.62064
Digital Object Identifier: doi:10.1214/07-AOS559
Project Euclid: euclid.aos/1231165183 - El Karoui, N. (2008b). Spectrum estimation for large dimensional covariance matrices using random matrix theory. Ann. Statist. 36 2757–2790.Mathematical Reviews (MathSciNet): MR2485012
Zentralblatt MATH: 1168.62052
Digital Object Identifier: doi:10.1214/07-AOS581
Project Euclid: euclid.aos/1231165184 - Fan, J., Feng, Y. and Wu, Y. (2009). Network exploration via the adaptive LASSO and SCAD penalties. Ann. Appl. Statist. 3 521–541.Mathematical Reviews (MathSciNet): MR2750671
Zentralblatt MATH: 1166.62040
Digital Object Identifier: doi:10.1214/08-AOAS215
Project Euclid: euclid.aoas/1245676184 - Fan, J., Huang, T. and Li, R. (2007). Analysis of longitudinal data with semiparametric estimation of convariance function. J. Amer. Statist. Assoc. 102 632–641.Mathematical Reviews (MathSciNet): MR2370857
Digital Object Identifier: doi:10.1198/016214507000000095 - Fan, J. and Lv, J. (2010). A selective overview of variable selection in high dimensional feature space. Statist. Sinica 20 101–148.
- Fitzmaurice, G., Davidian, M., Verbeke, G. and Molenberghs, G., eds. (2009). Longitudinal Data Analysis. CRC Press, Boca Raton, FL.Mathematical Reviews (MathSciNet): MR1500110
- Flury, B. (1988). Common Principal Components and Related Multivariate Models. Wiley, New York.
- Friedman, J., Hastie, T. and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9 432–441.
- Friedman, J., Hastie, T. and Tibshirani, R. (2010). Applications of the lasso and grouped lasso to the estimation of sparse graphical models. Technical report, Stanford Univ.
- Furrer, R. and Bengtsson, T. (2007). Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants. J. Multivariate Anal. 98 227–255.Mathematical Reviews (MathSciNet): MR2301751
Digital Object Identifier: doi:10.1016/j.jmva.2006.08.003 - Gabriel, K. R. (1962). Ante-dependence analysis of an ordered set of variables. Ann. Math. Statist. 33 201–212.Mathematical Reviews (MathSciNet): MR145611
Zentralblatt MATH: 0111.15604
Digital Object Identifier: doi:10.1214/aoms/1177704724
Project Euclid: euclid.aoms/1177704724 - Garthwaite, P. H. and Al-Awadhi, S. A. (2001). Non-conjugate prior distribution assessment for multivariate normal sampling. J. R. Stat. Soc. Ser. B Stat. Methodol. 63 95–110.Mathematical Reviews (MathSciNet): MR1811993
Zentralblatt MATH: 0976.62047
Digital Object Identifier: doi:10.1111/1467-9868.00278 - Golub, G. H. and Van Loan, C. F. (1989). Matrix Computations, 2nd ed. Johns Hopkins Series in the Mathematical Sciences 3. Johns Hopkins Univ. Press, Baltimore, MD.Mathematical Reviews (MathSciNet): MR1002570
- Haff, L. R. (1980). Empirical Bayes estimation of the multivariate normal covariance matrix. Ann. Statist. 8 586–597.Mathematical Reviews (MathSciNet): MR568722
Zentralblatt MATH: 0441.62045
Digital Object Identifier: doi:10.1214/aos/1176345010
Project Euclid: euclid.aos/1176345010 - Haff, L. R. (1991). The variational form of certain Bayes estimators. Ann. Statist. 19 1163–1190.Mathematical Reviews (MathSciNet): MR1126320
Zentralblatt MATH: 0739.62046
Digital Object Identifier: doi:10.1214/aos/1176348244
Project Euclid: euclid.aos/1176348244 - Hastie, T., Tibshirani, R. and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. Springer, New York.Mathematical Reviews (MathSciNet): MR2722294
- Hoff, P. D. (2009). A hierarchical eigenmodel for pooled covariance estimation. J. Roy. Statist. Soc. Ser. B 71 971–992.Mathematical Reviews (MathSciNet): MR2750253
Digital Object Identifier: doi:10.1111/j.1467-9868.2009.00716.x - Hoff, P. D. and Niu, X. (2009). A covariance regression model. Technical report, Univ. Washington.
- Huang, J. Z., Liu, L. and Liu, N. (2007). Estimation of large covariance matrices of longitudinal data with basis function approximations. J. Comput. Graph. Statist. 16 189–209.Mathematical Reviews (MathSciNet): MR2345752
Digital Object Identifier: doi:10.1198/106186007X181452 - Huang, J. Z., Liu, N., Pourahmadi, M. and Liu, L. (2006). Covariance matrix selection and estimation via penalised normal likelihood. Biometrika 93 85–98.Mathematical Reviews (MathSciNet): MR2277742
Zentralblatt MATH: 1152.62346
Digital Object Identifier: doi:10.1093/biomet/93.1.85 - James, W. and Stein, C. (1961). Estimation with quadratic loss. In Proc. 4th Berkeley Sympos. Math. Statist. Probab. I 361–379. Univ. California Press, Berkeley.Mathematical Reviews (MathSciNet): MR133191
- Jiang, G., Sarkar, S. K. and Hsuan, F. (1999). A likelihood ratio test and its modifications for the homogeneity of the covariance matrices of dependent multivariate normals. J. Statist. Plann. Inference 81 95–111.
- Joe, H. (2006). Generating random correlation matrices based on partial correlations. J. Multivariate Anal. 97 2177–2189.Mathematical Reviews (MathSciNet): MR2301633
Zentralblatt MATH: 1112.62055
Digital Object Identifier: doi:10.1016/j.jmva.2005.05.010 - Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295–327.Mathematical Reviews (MathSciNet): MR1863961
Zentralblatt MATH: 1016.62078
Digital Object Identifier: doi:10.1214/aos/1009210544
Project Euclid: euclid.aos/1009210544 - Johnstone, I. M. and Lu, A. Y. (2009). On consistency and sparsity for principal components analysis in high dimensions. J. Amer. Statist. Assoc. 104 682–693.
- Jones, R. H. (1980). Maximum likelihood fitting of ARMA models to time series with missing observations. Technometrics 22 389–395.
- Jones, M. C. (1987). Randomly choosing parameters from the stationarity and invertibility region of autoregressive-moving average models. J. Roy. Statist. Soc. Ser. C 36 134–138.Mathematical Reviews (MathSciNet): MR897452
Zentralblatt MATH: 0617.62100
Digital Object Identifier: doi:10.2307/2347544 - Jong, J.-C. and Kotz, S. (1999). On a relation between principal components and regression analysis. Amer. Statist. 53 349–351.
- Kalman, A. E. (1960). A new approach to linear filtering and prediction problems. Trans. Amer. Soc. Mech. Eng.—J. Basic Engineering 82 35–45.
- Kaufman, C. G., Schervish, M. J. and Nychka, W. (2008). Covariance tapering for likelihood-based estimation in large data sets. J. Amer. Statist. Assoc. 103 145–155.
- Kurowicka, D. and Cooke, R. (2003). A parameterization of positive definite matrices in terms of partial correlation vines. Linear Algebra Appl. 372 225–251.Mathematical Reviews (MathSciNet): MR1999149
Zentralblatt MATH: 1027.60070
Digital Object Identifier: doi:10.1016/S0024-3795(03)00507-X - Lam, C. and Fan, J. (2009). Sparsistency and rates of convergence in large covariance matrix estimation. Ann. Statist. 37 4254–4278.Mathematical Reviews (MathSciNet): MR2572459
Zentralblatt MATH: 1191.62101
Digital Object Identifier: doi:10.1214/09-AOS720
Project Euclid: euclid.aos/1256303543 - Ledoit, O., Santa-Clara, P. and Wolf, M. (2003). Flexible multivariate GARCH modeling with an application to international stock markets. Rev. Econom. Statist. 85 735–747.
- Ledoit, O. and Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88 365–411.Mathematical Reviews (MathSciNet): MR2026339
Zentralblatt MATH: 1032.62050
Digital Object Identifier: doi:10.1016/S0047-259X(03)00096-4 - Leng, C., Zhang, W. and Pan, J. (2010). Semiparametric mean-covariance regression analysis for longitudinal data. J. Amer. Statist. Assoc. 105 181–193.Mathematical Reviews (MathSciNet): MR2656048
Digital Object Identifier: doi:10.1198/jasa.2009.tm08485 - Leonard, T. and Hsu, J. S. J. (1992). Bayesian inference for a covariance matrix. Ann. Statist. 20 1669–1696.Mathematical Reviews (MathSciNet): MR1193308
Zentralblatt MATH: 0765.62031
Digital Object Identifier: doi:10.1214/aos/1176348885
Project Euclid: euclid.aos/1176348885 - LeSage, J. P. and Pace, R. K. (2007). A matrix exponential spatial specification. J. Econometrics 140 190–214.Mathematical Reviews (MathSciNet): MR2395921
Digital Object Identifier: doi:10.1016/j.jeconom.2006.09.007 - Leung, P. L. and Muirhead, R. J. (1987). Estimation of parameter matrices and eigenvalues in MANOVA and canonical correlation analysis. Ann. Statist. 15 1651–1666.Mathematical Reviews (MathSciNet): MR913580
Zentralblatt MATH: 0629.62059
Digital Object Identifier: doi:10.1214/aos/1176350616
Project Euclid: euclid.aos/1176350616 - Levina, E., Rothman, A. and Zhu, J. (2008). Sparse estimation of large covariance matrices via a nested Lasso penalty. Ann. Appl. Statist. 2 245–263.Mathematical Reviews (MathSciNet): MR2415602
Zentralblatt MATH: 1137.62338
Digital Object Identifier: doi:10.1214/07-AOAS139
Project Euclid: euclid.aoas/1206367820 - Liang, K. Y. and Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika 73 13–22.Mathematical Reviews (MathSciNet): MR836430
Zentralblatt MATH: 0595.62110
Digital Object Identifier: doi:10.1093/biomet/73.1.13 - Liechty, J. C., Liechty, M. W. and Müller, P. (2004). Bayesian correlation estimation. Biometrika 91 1–14.Mathematical Reviews (MathSciNet): MR2050456
Zentralblatt MATH: 1132.62314
Digital Object Identifier: doi:10.1093/biomet/91.1.1 - Lin, T. I. (2011). A Bayesian inference in joint modelling of location and scale parameters of the t distribution for longitudinal data. J. Statist. Plann. Inference 141 1543–1553.Mathematical Reviews (MathSciNet): MR2747923
Zentralblatt MATH: 1204.62040
Digital Object Identifier: doi:10.1016/j.jspi.2010.11.001 - Lin, S. P. and Perlman, M. D. (1985). A Monte Carlo comparison of four estimators of a covariance matrix. In Multivariate Analysis VI (Pittsburgh, PA, 1983) 411–429. North-Holland, Amsterdam.
- Lin, T.-I. and Wang, Y.-J. (2009). A robust approach to joint modeling of mean and scale covariance for longitudinal data. J. Statist. Plann. Inference 139 3013–3026.Mathematical Reviews (MathSciNet): MR2535179
Zentralblatt MATH: 1168.62082
Digital Object Identifier: doi:10.1016/j.jspi.2009.02.008 - Liu, C. (1993). Bartlett’s decomposition of the posterior distribution of the covariance for normal monotone ignorable missing data. J. Multivariate Anal. 46 198–206.Mathematical Reviews (MathSciNet): MR1240420
Zentralblatt MATH: 0778.62004
Digital Object Identifier: doi:10.1006/jmva.1993.1056 - Liu, X. and Daniels, M. J. (2006). A new algorithm for simulating a correlation matrix based on parameter expansion and reparameterization. J. Comput. Graph. Statist. 15 897–914.Mathematical Reviews (MathSciNet): MR2297634
Digital Object Identifier: doi:10.1198/106186006X160681 - McMurry, T. L. and Politis, D. N. (2010). Banded and tapered estimates for autocovariance matrices and the linear process bootstrap. J. Time Series Anal. 31 471–482.Mathematical Reviews (MathSciNet): MR2732601
Digital Object Identifier: doi:10.1111/j.1467-9892.2010.00679.x - Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. Ann. Statist. 34 1436–1462.Mathematical Reviews (MathSciNet): MR2278363
Zentralblatt MATH: 1113.62082
Digital Object Identifier: doi:10.1214/009053606000000281
Project Euclid: euclid.aos/1152540754 - Pan, J. and Mackenzie, G. (2003). On modelling mean-covariance structures in longitudinal studies. Biometrika 90 239–244.Mathematical Reviews (MathSciNet): MR1966564
Zentralblatt MATH: 1039.62068
Digital Object Identifier: doi:10.1093/biomet/90.1.239 - Peng, J., Zhou, N. and Zhu, J. (2009). Partial correlation estimation by joint sparse regression models. J. Amer. Statist. Assoc. 104 735–746.
- Pinheiro, J. D. and Bates, D. M. (1996). Unconstrained parameterizations for variance–covariance matrices. Stat. Comput. 6 289–366.
- Pourahmadi, M. (1999). Joint mean-covariance models with applications to longitudinal data: Unconstrained parameterisation. Biometrika 86 677–690.Mathematical Reviews (MathSciNet): MR1723786
Zentralblatt MATH: 0949.62066
Digital Object Identifier: doi:10.1093/biomet/86.3.677 - Pourahmadi, M. (2000). Maximum likelihood estimation of generalised linear models for multivariate normal covariance matrix. Biometrika 87 425–435.
- Pourahmadi, M. (2001). Foundations of Time Series Analysis and Prediction Theory. Wiley, New York.
- Pourahmadi, M. (2007a). Cholesky decompositions and estimation of a multivariate normal covariance matrix: Parameter orthogonality. Biometrika 94 1006–1013.
- Pourahmadi, M. (2007b). Simultaneous modeling of covariance matrices: GLM, Bayesian and nonparametric perspective. In Correlated Data Modelling 2004 (D. Gregori et al., eds.) 41–64. FrancoAngeli, Milan, Italy.
- Pourahmadi, M. and Daniels, M. J. (2002). Dynamic conditionally linear mixed models for longitudinal data. Biometrics 58 225–231.Mathematical Reviews (MathSciNet): MR1891383
Digital Object Identifier: doi:10.1111/j.0006-341X.2002.00225.x - Quenouille, M. H. (1949). Approximate tests of correlation in time-series. J. Roy. Statist. Soc. Ser. B 11 68–84.Mathematical Reviews (MathSciNet): MR32176
- Rajaratnam, B., Massam, H. and Carvalho, C. M. (2008). Flexible covariance estimation in graphical Gaussian models. Ann. Statist. 36 2818–2849.Mathematical Reviews (MathSciNet): MR2485014
Zentralblatt MATH: 1168.62054
Digital Object Identifier: doi:10.1214/08-AOS619
Project Euclid: euclid.aos/1231165186 - Ramsey, F. L. (1974). Characterization of the partial autocorrelation function. Ann. Statist. 2 1296–1301.Mathematical Reviews (MathSciNet): MR359219
Zentralblatt MATH: 0301.62046
Digital Object Identifier: doi:10.1214/aos/1176342881
Project Euclid: euclid.aos/1176342881 - Rocha, G. V., Zhao, P. and Yu, B. (2008). A path following algorithm for sparse pseudo-likelihood inverse covariance estimation (splice). Technical Report 759, Dept. Statistics, Univ. California, Berkeley.
- Rothman, A. J., Levina, E. and Zhu, J. (2009). Generalized thresholding of large covariance matrices. J. Amer. Statist. Assoc. 104 177–186.
- Rothman, A. J., Levina, E. and Zhu, J. (2010). A new approach to Cholesky-based covariance regularization in high dimensions. Biometrika 97 539–550.Mathematical Reviews (MathSciNet): MR2672482
Zentralblatt MATH: 1195.62089
Digital Object Identifier: doi:10.1093/biomet/asq022 - Rothman, A. J., Bickel, P. J., Levina, E. and Zhu, J. (2008). Sparse permutation invariant covariance estimation. Electron. J. Stat. 2 494–515.Mathematical Reviews (MathSciNet): MR2417391
Digital Object Identifier: doi:10.1214/08-EJS176
Project Euclid: euclid.ejs/1214491853 - Roy, J. (1958). Step-down procedure in multivariate analysis. Ann. Math. Statist. 29 1177–1187.Mathematical Reviews (MathSciNet): MR100938
Zentralblatt MATH: 0087.33907
Digital Object Identifier: doi:10.1214/aoms/1177706449
Project Euclid: euclid.aoms/1177706449 - Searle, S. R., Casella, G. and McCulloch, C. E. (1992). Variance Components. Wiley, New York.Mathematical Reviews (MathSciNet): MR1190470
- Smith, M. and Kohn, R. (2002). Parsimonious covariance matrix estimation for longitudinal data. J. Amer. Statist. Assoc. 97 1141–1153.Mathematical Reviews (MathSciNet): MR1951266
Zentralblatt MATH: 1041.62044
Digital Object Identifier: doi:10.1198/016214502388618942 - Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. In Proc. Third Berkeley Symp. Math. Statist. Probab. I 197–206. Univ. California Press, Berkeley.
- Stein, C. (1975). Estimation of a covariance matrix. In Rietz Lecture. 39th Annual Meeting IMS. Atlanta, Georgia.
- Szatrowski, T. H. (1980). Necessary and sufficient conditions for explicit solutions in the multivariate normal estimation problem for patterned means and covariances. Ann. Statist. 8 802–810.Mathematical Reviews (MathSciNet): MR572623
Digital Object Identifier: doi:10.1214/aos/1176345072
Project Euclid: euclid.aos/1176345072 - Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267–288.Mathematical Reviews (MathSciNet): MR1379242
- Wagaman, A. S. and Levina, E. (2009). Discovering sparse covariance structures with the Isomap. J. Comput. Graph. Statist. 18 551–572.
- Warton, D. I. (2008). Penalized normal likelihood and ridge regularization of correlation and covariance matrices. J. Amer. Statist. Assoc. 103 340–349.Mathematical Reviews (MathSciNet): MR2394637
Zentralblatt MATH: 05564493
Digital Object Identifier: doi:10.1198/016214508000000021 - Wermuth, N. (1980). Linear recursive equations, covariance selection, and path analysis. J. Amer. Statist. Assoc. 75 963–972.Mathematical Reviews (MathSciNet): MR600984
Zentralblatt MATH: 0475.62056
Digital Object Identifier: doi:10.2307/2287189 - Witten, D. M. and Tibshirani, R. (2009). Covariance-regularized regression and classification for high-dimensional problems. J. Roy. Statist. Soc. Ser. B 71 615–636.Mathematical Reviews (MathSciNet): MR2749910
Digital Object Identifier: doi:10.1111/j.1467-9868.2009.00699.x - Wold, H. O. A. (1960). A generalization of causal chain models. Econometrica 28 443–463.
- Wong, F., Carter, C. K. and Kohn, R. (2003). Efficient estimation of covariance selection models. Biometrika 90 809–830.
- Wright, S. (1934). The method of path coefficients. Ann. Math. Statist. 5 161–215.
- Wu, W. B. and Pourahmadi, M. (2003). Nonparametric estimation of large covariance matrices of longitudinal data. Biometrika 90 831–844.
- Wu, W. B. and Pourahmadi, M. (2009). Banding sample autocovariance matrices of stationary processes. Statist. Sinica 19 1755–1768.
- Yang, R.-Y. and Berger, J. O. (1994). Estimation of a covariance matrix using the reference prior. Ann. Statist. 22 1195–1211.Mathematical Reviews (MathSciNet): MR1311972
Zentralblatt MATH: 0819.62013
Digital Object Identifier: doi:10.1214/aos/1176325625
Project Euclid: euclid.aos/1176325625 - Yuan, M. and Huang, J. Z. (2009). Regularized parameter estimation of high dimensional t distribution. J. Statist. Plann. Inference 139 2284–2292.Mathematical Reviews (MathSciNet): MR2507990
Zentralblatt MATH: 1160.62053
Digital Object Identifier: doi:10.1016/j.jspi.2008.10.014 - Yuan, M. and Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika 94 19–35.Mathematical Reviews (MathSciNet): MR2367824
Zentralblatt MATH: 1142.62408
Digital Object Identifier: doi:10.1093/biomet/asm018 - Yule, G. U. (1907). On the theory of correlation for any number of variables, treated by a new system of notation. Roy. Soc. Proc. 79 85–96.
- Yule, G. U. (1927). On a model of investigating periodicities in disturbed series with special reference to Wolfer’s sunspot numbers. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 226 267–298.
- Zimmerman, D. L. (2000). Viewing the correlation structure of longitudinal data through a PRISM. Amer. Statist. 54 310–318.
- Zimmerman, D. L. and Núñez-Antón, V. (2001). Parametric modelling of growth curve data: An overview (with discussion). Test 10 1–73.Mathematical Reviews (MathSciNet): MR1856193
Zentralblatt MATH: 0981.62050
Digital Object Identifier: doi:10.1007/BF02595823 - Zimmerman, D. L. and Núñez-Antón, V. A. (2010). Antedependence Models for Longitudinal Data. Monographs on Statistics and Applied Probability 112. CRC Press, Boca Raton, FL.
- Zou, H., Hastie, T. and Tibshirani, R. (2006). Sparse principal component analysis. J. Comput. Graph. Statist. 15 265–286.Mathematical Reviews (MathSciNet): MR2252527
Digital Object Identifier: doi:10.1198/106186006X113430
- You have access to this content.
- You have partial access to this content.
- You do not have access to this content.
More like this
- Posterior convergence rates for estimating large precision matrices using graphical models
Banerjee, Sayantan and Ghosal, Subhashis, Electronic Journal of Statistics, 2014 - Sparse estimation of large covariance matrices
via a nested Lasso penalty
Levina, Elizaveta, Rothman, Adam, and Zhu, Ji, The Annals of Applied Statistics, 2008 - High dimensional sparse covariance estimation via directed acyclic graphs
Rütimann, Philipp and Bühlmann, Peter, Electronic Journal of Statistics, 2009
- Posterior convergence rates for estimating large precision matrices using graphical models
Banerjee, Sayantan and Ghosal, Subhashis, Electronic Journal of Statistics, 2014 - Sparse estimation of large covariance matrices
via a nested Lasso penalty
Levina, Elizaveta, Rothman, Adam, and Zhu, Ji, The Annals of Applied Statistics, 2008 - High dimensional sparse covariance estimation via directed acyclic graphs
Rütimann, Philipp and Bühlmann, Peter, Electronic Journal of Statistics, 2009 - Adaptive estimation of covariance matrices via Cholesky decomposition
Verzelen, Nicolas, Electronic Journal of Statistics, 2010 - High dimensional posterior convergence rates for decomposable graphical models
Xiang, Ruoxuan, Khare, Kshitij, and Ghosh, Malay, Electronic Journal of Statistics, 2015 - Fast global convergence of gradient methods for high-dimensional statistical recovery
Agarwal, Alekh, Negahban, Sahand, and Wainwright, Martin J., The Annals of Statistics, 2012 - Non-Euclidean statistics for covariance
matrices, with applications to diffusion tensor imaging
Dryden, Ian L., Koloydenko, Alexey, and Zhou, Diwei, The Annals of Applied Statistics, 2009 - Sparsistency and rates of convergence in large covariance matrix estimation
Lam, Clifford and Fan, Jianqing, The Annals of Statistics, 2009 - Innovated scalable efficient estimation in ultra-large Gaussian graphical models
Fan, Yingying and Lv, Jinchi, The Annals of Statistics, 2016 - Minimax posterior convergence rates and model selection consistency in high-dimensional DAG models based on sparse Cholesky factors
Lee, Kyoungjae, Lee, Jaeyong, and Lin, Lizhen, The Annals of Statistics, 2019