Communications in Applied Mathematics and Computational Science

On a scalable nonparametric denoising of time series signals

Lukáš Pospíšil, Patrick Gagliardini, William Sawyer, and Illia Horenko

Full-text: Access denied (no subscription detected)

However, an active subscription may be available with MSP at

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


Denoising and filtering of time series signals is a problem emerging in many areas of computational science. Here we demonstrate how the nonparametric computational methodology of the finite element method of time series analysis with H 1 regularization can be extended for denoising of very long and noisy time series signals. The main computational bottleneck is the inner quadratic programming problem. Analyzing the solvability and utilizing the problem structure, we suggest an adapted version of the spectral projected gradient method (SPG-QP) to resolve the problem. This approach increases the granularity of parallelization, making the proposed methodology highly suitable for graphics processing unit (GPU) computing. We demonstrate the scalability of our open-source implementation based on PETSc for the Piz Daint supercomputer of the Swiss Supercomputing Centre (CSCS) by solving large-scale data denoising problems and comparing their computational scaling and performance to the performance of the standard denoising methods.

Article information

Commun. Appl. Math. Comput. Sci., Volume 13, Number 1 (2018), 107-138.

Received: 20 June 2017
Accepted: 30 October 2017
First available in Project Euclid: 28 March 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 37M10: Time series analysis 62-07: Data analysis 62H30: Classification and discrimination; cluster analysis [See also 68T10, 91C20] 65Y05: Parallel computation 90C20: Quadratic programming

time series analysis quadratic programming SPG-QP regularization


Pospíšil, Lukáš; Gagliardini, Patrick; Sawyer, William; Horenko, Illia. On a scalable nonparametric denoising of time series signals. Commun. Appl. Math. Comput. Sci. 13 (2018), no. 1, 107--138. doi:10.2140/camcos.2018.13.107.

Export citation


  • N. Agmon, Y. Alhassid, and R. D. Levine, An algorithm for finding the distribution of maximal entropy, J. Comput. Phys. 30 (1979), no. 2, 250–258.
  • S. Balay, S. Abhyankar, M. F. Adams, J. Brown, P. Brune, K. Buschelman, L. Dalcin, V. Eijkhout, W. D. Gropp, D. Kaushik, M. G. Knepley, D. A. May, L. C. McInnes, K. Rupp, B. F. Smith, S. Zampini, H. Zhang, and H. Zhang, PETSc, users manual ANL-95/11, rev. 3.7, Argonne National Laboratory, 2016.
  • J. Barzilai and J. M. Borwein, Two-point step size gradient methods, IMA J. Numer. Anal. 8 (1988), no. 1, 141–148.
  • D. P. Bertsekas, Nonlinear programming, 2nd ed., Athena Scientific, 1999.
  • E. G. Birgin, J. M. Martínez, and M. Raydan, Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optim. 10 (2000), no. 4, 1196–1211.
  • ––––, Algorithm 813: SPG - software for convex-constrained optimization, ACM Trans. Math. Softw. 27 (2001), no. 3, 340–349.
  • S. Boyd and L. Vandenberghe, Convex optimization, Cambridge University, 2004.
  • E. J. Candès and M. B. Wakin, An introduction to compressive sampling, IEEE Signal Proc. Mag. 25 (2008), no. 2, 21–30.
  • A.-L. Cauchy, Méthode générale pour la résolution des systèmes d'équations simultanées, Comptes Rendus Hebd. Séances Acad. Sci. 25 (1847), 536–538.
  • Y. Chen and X. Ye, Projection onto a simplex, preprint, 2011.
  • A. Coates, B. Huval, T. Wang, D. J. Wu, A. Y. Ng, and B. Catanzaro, Deep learning with COTS HPC systems, Proceedings of the 30th International Conference on Machine Learning (S. Dasgupta and D. McAllester, eds.), Proceedings of Machine Learning Research, no. 28, 2013, pp. 1337–1345.
  • Y.-H. Dai and R. Fletcher, Projected Barzilai–Borwein methods for large-scale box-constrained quadratic programming, Numer. Math. 100 (2005), no. 1, 21–47.
  • Y.-H. Dai and L.-Z. Liao, $R$-linear convergence of the Barzilai and Borwein gradient method, IMA J. Numer. Anal. 22 (2002), no. 1, 1–10.
  • Z. Dostál, Optimal quadratic programming algorithms: with applications to variational inequalities, Springer Optimization and Its Applications, no. 23, Springer, 2009.
  • Z. Dostál and L. Pospíšil, Minimizing quadratic functions with semidefinite Hessian subject to bound constraints, Comput. Math. Appl. 70 (2015), no. 8, 2014–2028.
  • ––––, Optimal iterative QP and QPQC algorithms, Ann. Oper. Res. 243 (2016), no. 1–2, 5–18.
  • W. Enders, Applied econometric time series, 4th ed., Wiley, 2015.
  • G. Fourestey, B. Cumming, L. Gilly, and T. C. Schulthess, First experiences with validating and using the Cray power management database tool, Cray User Group proceedings, 2014.
  • S. Gerber and I. Horenko, On inference of causality for discrete state models in a multiscale context, Proc. Natl. Acad. Sci. USA 111 (2014), no. 41, 14651–14656.
  • ––––, Improving clustering by imposing network information, Sci. Adv. 1 (2015), no. 7, 1500163.
  • L. Grippo, F. Lampariello, and S. Lucidi, A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal. 23 (1986), no. 4, 707–716.
  • P. C. Hansen and D. P. O'Leary, The use of the $L$-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput. 14 (1993), no. 6, 1487–1503.
  • V. Hapla, Massively parallel quadratic programming solvers with applications in mechanics, Ph.D. thesis, Technical University of Ostrava, 2016.
  • V. Hapla, D. Horák, L. Pospíšil, M. Čermák, A. Vašatová, and R. Sojka, Solving contact mechanics problems with PERMON, High performance computing in science and engineering: second international conference (T. Kozubek, R. Blaheta, J. Šístek, M. Rozložník, and M. Čermák, eds.), Lecture Notes in Computer Science, no. 9611, Springer, 2015, pp. 101–115.
  • T. J. Hastie and R. J. Tibshirani, Generalized additive models, Monographs on Statistics and Applied Probability, no. 43, Chapman and Hall, 1990.
  • M. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Research Nat. Bur. Standards 49 (1952), 409–436.
  • I. Horenko, On simultaneous data-based dimension reduction and hidden phase identification, J. Atmos. Sci. 65 (2008), no. 6, 1941–1954.
  • ––––, On robust estimation of low-frequency variability trends in discrete Markovian sequences of atmospheric circulation patterns, J. Atmos. Sci. 66 (2009), no. 7, 2059–2072.
  • ––––, Finite element approach to clustering of multidimensional time series, SIAM J. Sci. Comput. 32 (2010), no. 1, 62–83.
  • ––––, On clustering of non-stationary meteorological time series, Dynam. Atmos. Oceans 49 (2010), no. 2–3, 164–187.
  • ––––, On the identification of nonstationary factor models and their application to atmospheric data analysis, J. Atmos. Sci. 67 (2010), no. 5, 1559–1574.
  • ––––, Nonstationarity in multifactor models of discrete jump processes, memory, and application to cloud modeling, J. Atmos. Sci. 68 (2011), no. 7, 1493–1506.
  • ––––, On analysis of nonstationary categorical data time series: dynamical dimension reduction, model selection, and applications to computational sociology, Multiscale Model. Simul. 9 (2011), no. 4, 1700–1726.
  • D. E. King, Dlib-ml: a machine learning toolkit, J. Mach. Learn. Res. 10 (2009), 1755–1758.
  • A. J. Laub, Matrix analysis for scientists and engineers, Society for Industrial and Applied Mathematics, 2005.
  • F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, and B. Arnaldi, A review of classification algorithms for EEG-based brain–computer interfaces, J. Neural Eng. 4 (2007), no. 2, R1.
  • J. H. Macke, M. Opper, and M. Bethge, Common input explains higher-order correlations and entropy in a simple model of neural population activity, Phys. Rev. Lett. 106 (2011), no. 20, 208102.
  • P. Metzner, L. Putzig, and I. Horenko, Analysis of persistent nonstationary time series and applications, Commun. Appl. Math. Comput. Sci. 7 (2012), no. 2, 175–229.
  • M. Mudelsee, Climate time series analysis: classical statistical and bootstrap methods, 2nd ed., Atmospheric and Oceanographic Sciences Library, no. 51, Springer, 2014.
  • T. Munson, J. Sarich, S. Wild, S. Benson, and L. C. McInnes, Toolkit for Advanced Optimization $($TAO$)$, users manual ANL/MCS-TM-322, rev. 3.5, Argonne National Laboratory, 2014.
  • K. Murphy, Hidden Markov model $($HMM$)$ toolbox for Matlab, 2005.
  • J. Nocedal and S. J. Wright, Numerical optimization, Springer, 1999.
  • L. Pospíšil, Development of algorithms for solving minimizing problems with convex quadratic function on special convex sets and applications, Ph.D. thesis, Technical University of Ostrava, 2015.
  • L. Pospíšil, I. Horenko, P. Gagliardini, and W. Sawyer, PASC-inference: open-source HPC library of nonparametric Bayesian causality inference methods, 2017.
  • M. Raydan M., Convergence properties of the Barzilai and Borwein gradient method, Ph.D. thesis, Rice University, 1991.
  • K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, preprint, 2015.
  • F. Stimberg, M. Opper, G. Sanguinetti, and A. Ruttor, Inference in continuous-time change-point models, Neural Information Processing Systems Conference 2011 (J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, eds.), Advances in Neural Information Processing Systems, no. 24, 2011, pp. 2717–2725.
  • L. A. Vese and C. Le Guyader, Variational methods in image processing, CRC, 2016.
  • G. Wahba, Spline models for observational data, CBMS-NSF Regional Conference Series in Applied Mathematics, no. 59, Society for Industrial and Applied Mathematics, 1990.
  • Y. Wiaux, L. Jacques, G. Puy, A. M. M. Scaife, and P. Vandergheynst, Compressed sensing imaging techniques for radio interferometry, Mon. Not. R. Astron. Soc. 395 (2009), no. 3, 1733–1742.
  • S. J. Wright, Primal-dual interior-point methods, Society for Industrial and Applied Mathematics, 1997.