Electronic Journal of Statistics

Matrix factorization for multivariate time series analysis

Pierre Alquier and Nicolas Marie

Full-text: Open access

Abstract

Matrix factorization is a powerful data analysis tool. It has been used in multivariate time series analysis, leading to the decomposition of the series in a small set of latent factors. However, little is known on the statistical performances of matrix factorization for time series. In this paper, we extend the results known for matrix estimation in the i.i.d setting to time series. Moreover, we prove that when the series exhibit some additional structure like periodicity or smoothness, it is possible to improve on the classical rates of convergence.

Article information

Source
Electron. J. Statist., Volume 13, Number 2 (2019), 4346-4366.

Dates
Received: March 2019
First available in Project Euclid: 6 November 2019

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1573009449

Digital Object Identifier
doi:10.1214/19-EJS1630

Mathematical Reviews number (MathSciNet)
MR4028508

Zentralblatt MATH identifier
07136618

Subjects
Primary: 62M20: Prediction [See also 60G25]; filtering [See also 60G35, 93E10, 93E11]
Secondary: 62H25: Factor analysis and principal components; correspondence analysis 62H12: Estimation 62M10: Time series, auto-correlation, regression, etc. [See also 91B84] 62G08: Nonparametric regression 93E14: Data smoothing 60G35: Signal detection and filtering [See also 62M20, 93E10, 93E11, 94Axx] 60B20: Random matrices (probabilistic aspects; for algebraic aspects see 15B52)

Keywords
Multivariate Time Series Analysis matrix Factorization random Matrices non-parametric Regression

Rights
Creative Commons Attribution 4.0 International License.

Citation

Alquier, Pierre; Marie, Nicolas. Matrix factorization for multivariate time series analysis. Electron. J. Statist. 13 (2019), no. 2, 4346--4366. doi:10.1214/19-EJS1630. https://projecteuclid.org/euclid.ejs/1573009449


Export citation

References

  • [1] P. Alquier. Bayesian methods for low-rank matrix estimation: short survey and theoretical study. In, International Conference on Algorithmic Learning Theory, pages 309–323. Springer, 2013.
  • [2] P. Alquier, V. Cottet, and G. Lecué. Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions., arXiv preprint, to appear in the Annals of Statistics, 2017.
  • [3] P. Alquier and P. Doukhan. Sparsity considerations for dependent variables., Electronic journal of statistics, 5:750–774, 2011.
  • [4] P. Alquier and B. Guedj. An oracle inequality for quasi-Bayesian nonnegative matrix factorization., Mathematical Methods of Statistics, 26(1):55–67, 2017.
  • [5] L. Bauwens and M. Lubrano. Identification restriction and posterior densities in cointegrated Gaussian VAR systems. In T. M. Fomby and R. Carter Hill, editors, Advances in econometrics, vol. 11(B). JAI Press, Greenwich, 1993.
  • [6] S. Boucheron, G. Lugosi, and P. Massart., Concentration inequalities: A nonasymptotic theory of independence. Oxford university press, 2013.
  • [7] T. Cai, D. Kim, Y. Wang, M. Yuan, and H. Zhou. Optimal large-scale quantum state tomography with Pauli measurements., The Annals of Statistics, 44(2):682–712, 2016.
  • [8] T. Cai and A. Zhang. Rop: Matrix recovery via rank-one projections., The Annals of Statistics, 43(1):102–138, 2015.
  • [9] E. J. Candes and Y. Plan. Matrix completion with noise., Proceedings of the IEEE, 98(6):925–936, 2010.
  • [10] E. J. Candès and B. Recht. Exact matrix completion via convex optimization., Foundations of Computational mathematics, 9(6):717, 2009.
  • [11] E. J. Candès and T. Tao. The power of convex relaxation: Near-optimal matrix completion., IEEE Transactions on Information Theory, 56(5) :2053–2080, 2010.
  • [12] L. Carel and P. Alquier. Non-negative matrix factorization as a pre-processing tool for travelers temporal profiles clustering. In, Proceedings of the 25th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pages 417–422, 2017.
  • [13] D. Chafaï, O. Guédon, G. Lecué, and A. Pajor., Interactions between compressed sensing random matrices and high dimensional geometry. Société Mathématique de France, 2012.
  • [14] V. Cheung, K. Devarajan, G. Severini, A. Turolla, and P. Bonato. Decomposing time series data by a non-negative matrix factorization algorithm with temporally constrained coefficients. In, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 3496–3499. IEEE, 2015.
  • [15] S. Chrétien and B. Guedj. Revisiting clustering as matrix factorisation on the Stiefel manifold., arXiv preprint arXiv :1903.04479, 2019.
  • [16] A. S. Dalalyan. Exponential weights in multivariate regression and a low-rankness favoring prior., arXiv preprint arXiv :1806.09405, 2018.
  • [17] A. S. Dalalyan, E. Grappin, and Q. Paris. On the exponentially weighted aggregate with the Laplace prior., The Annals of Statistics, 46(5) :2452–2478, 2018.
  • [18] Y. De Castro, Y. Goude, G. Hébrail, and J. Mei. Recovering multiple nonnegative time series from a few temporal aggregates. In, ICML 2017-34th International Conference on Machine Learning, pages 1–9, 2017.
  • [19] J. Dedecker, P. Doukhan, G. Lang, L. R. J. Rafael, S. Louhichi, and C. Prieur., Weak dependence: With examples and applications. Springer, 2007.
  • [20] R. F. Engle and C. W. J. Granger. Co-integration and error correction: representation, estimation, and testing., Econometrica: journal of the Econometric Society, pages 251–276, 1987.
  • [21] I. A. Genevera, L. Grosenick, and J. Taylor. A generalized least-square matrix decomposition., Journal of the American Statistical Association, 109(505):145–159, 2014.
  • [22] J. Geweke. Bayesian reduced rank regression in econometrics., Journal of Econometrics, 75:121–146, 1996.
  • [23] D. Gross. Recovering low-rank matrices from few coefficients in any basis., Information Theory, IEEE Transactions on, 57(3) :1548–1566, 2011.
  • [24] S. Gultekin and J. Paisley. Online forecasting matrix factorization., arXiv preprint arXiv :1712.08734, 2017.
  • [25] M. Guţă, T. Kypraios, and I. Dryden. Rank-based model selection for multiple ions quantum tomography., New Journal of Physics, 14(10) :105002, 2012.
  • [26] F; Husson, J. Josse, B. Narasimhan, and G. Robin. Imputation of mixed data with multilevel singular value decomposition., arXiv preprint arXiv :1804.11087, 2018.
  • [27] A. Izenman. Reduced rank regression for the multivariate linear model., Journal of Multivariate Analysis, 5(2):248–264, 1975.
  • [28] F. Kleibergen and H. K. van Dijk. On the shape of the likelihood-posterior in cointegration models., Econometric theory, 10:514–551, 1994.
  • [29] F. Kleibergen and H. K. van Dijk. Bayesian simultaneous equation analysis using reduced rank structures., Econometric theory, 14:699–744, 1998.
  • [30] O. Klopp, K. Lounici, and A. B. Tsybakov. Robust matrix completion., Probability Theory and Related Fields, 169(1–2):523–564, 2017.
  • [31] O. Klopp, Y. Lu, A. B. Tsybakov, and H. H. Zhou. Structured matrix estimation and completion., arXiv preprint arXiv :1707.02090, 2017.
  • [32] V. Koltchinskii, K. Lounici, and A. B. Tsybakov. Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion., The Annals of Statistics, 39(5) :2302–2329, 2011.
  • [33] G. Koop and D. Korobilis. Bayesian multivariate time series methods for empirical macroeconomics., Foundations and Trends® in Econometrics, 3(4):267–358, 2010.
  • [34] Y. Koren, R. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems., Computer, 42(8):30–37, 2009.
  • [35] D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization., Nature, 401 (6755):788–791, 1999.
  • [36] A. Lumbreras, L. Filstroff, and C. Févotte. Bayesian mean-parameterized nonnegative binary matrix factorization., arXiv preprint arXiv :1812.06866, 2018.
  • [37] T. D. Luu, J. Fadili, and C. Chesneau. Sharp oracle inequalities for low-complexity priors., arXiv preprint arXiv :1702.03166, 2017.
  • [38] T. T. Mai and P. Alquier. A Bayesian approach for noisy matrix completion: Optimal rate under general sampling distribution., Electronic Journal of Statistics, 9(1):823–841, 2015.
  • [39] T. T. Mai and P. Alquier. Pseudo-Bayesian quantum tomography with rank-adaptation., Journal of Statistical Planning and Inference, 184:62–76, 2017.
  • [40] J. Mei, Y. De Castro, Y. Goude, J.-M. Azaïs, and G. Hébrail. Nonnegative matrix factorization with side information for time series recovery and prediction., IEEE Transactions on Knowledge and Data Engineering, 2018.
  • [41] K. Moridomi, K. Hatano, and E. Takimoto. Tighter generalization bounds for matrix completion via factorization into constrained matrices., IEICE Transactions on Information and Systems, 101(8) :1997–2004, 2018.
  • [42] A. Ozerov and C. Févotte. Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation., IEEE Transactions on Audio, Speech, and Language Processing, 18(3):550–563, 2010.
  • [43] J. Paisley, D. Blei, and M. I. Jordan., Bayesian nonnegative matrix factorization with stochastic variational inference, volume Handbook of Mixed Membership Models and Their Applications, chapter 11. Chapman and Hall/CRC, 2015.
  • [44] E. Richard, S. Gaïffas, and N. Vayatis. Link prediction in graphs with autoregressive features., The Journal of Machine Learning Research, 15(1):565–593, 2014.
  • [45] A. Saha and V. Sindhwani. Learning evolving and emerging topics in social media: a dynamic nmf approach with temporal regularization. In, Proceedings of the fifth ACM international conference on Web search and data mining, pages 693–702. ACM, 2012.
  • [46] F. Shahnaz, M. W. Berry, V. P. Pauca, and R. J. Plemmons. Document clustering using nonnegative matrix factorization., Information Processing & Management, 42(2):373–386, 2006.
  • [47] T. Suzuki. Convergence rate of Bayesian tensor estimator and its minimax optimality. In, International Conference on Machine Learning, pages 1273–1282, 2015.
  • [48] E. Tonnelier, N. Baskiotis, V. Guigue, and P. Gallinari. Anomaly detection in smart card logs and distant evaluation with twitter: a robust framework., Neurocomputing, 298:109–121, 2018.
  • [49] J. A. Tropp. User-friendly tail bounds for sums of random matrices., Foundations of computational mathematics, 12(4):389–434, 2012.
  • [50] A. B. Tsybakov., Introduction to Nonparametric Estimation. 2009.
  • [51] C. Vernade and O. Cappé. Learning from missing data using selection bias in movie recommendation. In, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pages 1–9. IEEE, 2015.
  • [52] R. Vershynin. Introduction to the non-asymptotic analysis of random matrices., arXiv preprint arXiv :1011.3027, 2010.
  • [53] D. Xia and V. Koltchinskii. Estimation of low rank density matrices: bounds in schatten norms and other distances., Electronic Journal of Statistics, 10(2) :2717–2745, 2016.
  • [54] H.-F. Yu, N. Rao, and I. S Dhillon. Temporal regularized matrix factorization for high-dimensional time series prediction. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 847–855. Curran Associates, Inc., 2016.