The Annals of Applied Statistics

Reactive point processes: A new approach to predicting power failures in underground electrical systems

Şeyda Ertekin, Cynthia Rudin, and Tyler H. McCormick

Full-text: Open access


Reactive point processes (RPPs) are a new statistical model designed for predicting discrete events in time based on past history. RPPs were developed to handle an important problem within the domain of electrical grid reliability: short-term prediction of electrical grid failures (“manhole events”), including outages, fires, explosions and smoking manholes, which can cause threats to public safety and reliability of electrical service in cities. RPPs incorporate self-exciting, self-regulating and saturating components. The self-excitement occurs as a result of a past event, which causes a temporary rise in vulner ability to future events. The self-regulation occurs as a result of an external inspection which temporarily lowers vulnerability to future events. RPPs can saturate when too many events or inspections occur close together, which ensures that the probability of an event stays within a realistic range. Two of the operational challenges for power companies are (i) making continuous-time failure predictions, and (ii) cost/benefit analysis for decision making and proactive maintenance. RPPs are naturally suited for handling both of these challenges. We use the model to predict power-grid failures in Manhattan over a short-term horizon, and to provide a cost/benefit analysis of different proactive maintenance programs.

Article information

Ann. Appl. Stat., Volume 9, Number 1 (2015), 122-144.

First available in Project Euclid: 28 April 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Point processes self-exciting processes energy grid reliability Bayesian analysis time-series


Ertekin, Şeyda; Rudin, Cynthia; McCormick, Tyler H. Reactive point processes: A new approach to predicting power failures in underground electrical systems. Ann. Appl. Stat. 9 (2015), no. 1, 122--144. doi:10.1214/14-AOAS789.

Export citation


  • Aït-Sahalia, Y., Cacho-Diaz, J. and Laeven, R. J. (2010). Modeling financial contagion using mutually exciting jump processes. Technical report, National Bureau of Economic Research, Cambridge, MA.
  • Bacry, E., Delattre, S., Hoffmann, M. and Muzy, J. F. (2013). Modelling microstructure noise with mutually exciting point processes. Quant. Finance 13 65–77.
  • Bartlett, M. S. (1963). The spectral analysis of point processes. J. R. Stat. Soc. Ser. B Stat. Methodol. 25 264–296.
  • Beaumont, M. A., Cornuet, J.-M., Marin, J.-M. and Robert, C. P. (2009). Adaptive approximate Bayesian computation. Biometrika 96 983–990.
  • Blundell, C., Beck, J. and Heller, K. A. (2012). Modelling reciprocating relationships with Hawkes processes. In Advances in Neural Information Processing Systems 25. Curran Associates, Red Hook, NY.
  • Chehrazi, N. and Weber, T. A. (2011). Dynamic valuation of delinquent credit-card accounts. Working paper.
  • Crane, R. and Sornette, D. (2008). Robust dynamic classes revealed by measuring the response function of a social system. Proc. Natl. Acad. Sci. USA 105 15649–15653.
  • Diggle, P. J. and Gratton, R. J. (1984). Monte Carlo methods of inference for implicit statistical models. J. R. Stat. Soc. Ser. B Stat. Methodol. 46 193–227.
  • DOE (2008). The Smart Grid, An Introduction. Technical report. Prepared by Litos Strategic Communication. US Dept. Energy, Office of Electricity Delivery & Energy Reliability.
  • Drovandi, C. C., Pettitt, A. N. and Faddy, M. J. (2011). Approximate Bayesian computation using indirect inference. J. R. Stat. Soc. Ser. C. Appl. Stat. 60 317–337.
  • Du, N., Song, L., Woo, H. and Zha, H. (2013). Uncover topic-sensitive information diffusion networks. In Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics 229–237.
  • Egesdal, M., Fathauer, C., Louie, K., Neuman, J., Mohler, G. and Lewis, E. (2010). Statistical and stochastic modeling of gang rivalries in Los Angeles. SIAM Undergraduate Research Online 3 72–94.
  • Embrechts, P., Liniger, T. and Lin, L. (2011). Multivariate Hawkes processes: An application to financial data. J. Appl. Probab. 48A 367–378.
  • Ertekin, S., Rudin, C. and McCormick, T. (2013). Predicting Power Failures with Reactive Point Processes. In AAAI Workshop on Late-Breaking Developments. AAAI Press, Menlo Park, CA.
  • Ertekin, S., Rudin, C. and McCormick, T. H. (2015). Supplement to “Reactive point processes: A new approach to predicting power failures in underground electrical systems.” DOI:10.1214/14-AOAS789SUPP.
  • Fearnhead, P. and Prangle, D. (2012). Constructing summary statistics for approximate Bayesian computation: Semi-automatic approximate Bayesian computation. J. R. Stat. Soc. Ser. B Stat. Methodol. 74 419–474.
  • Filimonov, V. and Sornette, D. (2012). Quantifying reflexivity in financial markets: Toward a prediction of flash crashes. Phys. Rev. E (3) 85 056108.
  • Guttorp, P. and Thorarinsdottir, T. L. (2012). Bayesian inference for non-Markovian point processes. In Advances and Challenges in Space–Time Modeling of Natural Events (E. Porcu, J.-M. Montero and M. Schlather, eds.) 79–102. Springer, Berlin.
  • Hardiman, S. J., Bercot, N. and Bouchaud, J.-P. (2013). Critical reflexivity in financial markets: A Hawkes process analysis. Eur. Phys. J. B 86 1–9.
  • Johnson, D. H. (1996). Point process models of single-neuron discharges. J. Comput. Neurosci. 3 275–299.
  • Kerstan, J. (1964). Teilprozesse Poissonscher Prozesse. In Trans. Third Prague Conf. Information Theory, Statist. Decision Functions, Random Processes (Liblice, 1962) 377–403. Publ. House Czech. Acad. Sci., Prague.
  • Krumin, M., Reutsky, I. and Shoham, S. (2010). Correlation-based analysis and generation of multiple spike trains using hawkes models with an exogenous input. Front. Comput. Neurosci. 4 147.
  • Lewis, E., Mohler, G., Brantingham, P. J. and Bertozzi, A. (2010). Self-exciting point process models of insurgency in Iraq. UCLA CAM Reports 10-38.
  • Louie, K., Masaki, M. and Allenby, M. (2010). A point process model for simulating gang-on-gang violence. Technical report, UCLA.
  • Masuda, N., Takaguchi, T., Sato, N. and Yano, K. (2012). Self-exciting point process modeling of conversation event sequences. Preprint. Available at arXiv:1205.5109.
  • Mitchell, L. and Cates, M. E. (2010). Hawkes process as a model of social interactions: A view on video dynamics. J. Phys. A 43 045101, 11.
  • Mohler, G. O., Short, M. B., Brantingham, P. J., Schoenberg, F. P. and Tita, G. E. (2011). Self-exciting point process modeling of crime. J. Amer. Statist. Assoc. 106 100–108.
  • NYBC (2010). Electricity OUTLOOK: Powering New York City’s economic future. Technical report. New York Building Congress Reports: Energy Outlook 2010–2025.
  • Ogata, Y. (1988). Statistical models for earthquake occurrences and residual analysis for point processes. J. Amer. Statist. Assoc. 83 9–27.
  • Ogata, Y. (1998). Space–time point-process models for earthquake occurrences. Ann. Inst. Statist. Math. 50 379–402.
  • Passonneau, R., Rudin, C., Radeva, A., Tomar, A. and Xie, B. (2011). Treatment effect of repairs to an electrical grid: Leveraging a machine learned model of structure vulnerability. In Proceedings of the KDD Workshop on Data Mining Applications in Sustainability (SustKDD), 17th Annual ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, New York.
  • Peruggia, M. and Santner, T. (1996). Bayesian analysis of time evolution of earthquakes. J. Amer. Statist. Assoc. 91 1209–1218.
  • Porter, M. D. and White, G. (2012). Self-exciting hurdle models for terrorist activity. Ann. Appl. Stat. 6 106–124.
  • Reynaud-Bouret, P. and Schbath, S. (2010). Adaptive estimation for Hawkes processes; application to genome analysis. Ann. Statist. 38 2781–2822.
  • Rhodes, L. (2013). US power grid has issues with reliability. Data Center Knowledge ( Industry Perspectives.
  • Rudin, C., Passonneau, R. J., Radeva, A., Dutta, H., Ierome, S. and Isaac, D. (2010). A process for predicting manhole events in Manhattan. Mach. Learn. 80 1–31.
  • Rudin, C., Waltz, D., Anderson, R. N., Boulanger, A., Salleb-Aouissi, A., Chow, M., Dutta, H., Gross, P. N., Huang, B., Ierome, S., Isaac, D. F., Kressner, A., Passonneau, R. J., Radeva, A. and Wu, L. (2012). Machine learning for the New York City power grid. IEEE Trans. Pattern. Anal. Mach. Intell. 34 328–345.
  • Rudin, C., Ertekin, S., Passonneau, R., Radeva, A., Tomar, A., Xie, B., Lewis, S., Riddle, M., Pangsrivinij, D. and McCormick, T. (2014). Analytics for power grid distribution reliability in New York City. Interfaces 44 364–383.
  • Simma, A. and Jordan, M. I. (2010). Modeling Events with Cascades of Poisson Processes. In Proc. of the 26th Conference on Uncertainty in Artificial Intelligence (UAI2010). AUAI Press.
  • So, H. (2004). Council approves bill on Con Ed annual inspections. The Villager 74 23.
  • Taddy, M. A. and Kottas, A. (2012). Mixture modeling for marked Poisson processes. Bayesian Anal. 7 335–361.

Supplemental materials

  • Supplementary material for “Reactive point processes: A new approach to predicting power failures in underground electrical systems”.: The supplementary material includes an expanded related work section, conditional frequency estimator (CF estimator) for the RPP, experiments with a maximum likelihood approach, a description of the inspection policy used in Section 6, an analysis of Manhattan data using random effects model and simulation studies for validating the fitting techniques for the models in the paper. It also includes a description and link for a publicly available simulated data set that we generated, based on statistical properties of the Manhattan data set.