Statistical Science

Filtering and Tracking Survival Propensity (Reconsidering the Foundations of Reliability)

Nozer D. Singpurwalla

Full-text: Open access


The work described here was motivated by the need to address a long standing problem in engineering, namely, the tracking of reliability growth. An archetypal scenario is the performance of software as it evolves over time. Computer scientists are challenged by the task of when to release software. The same is also true for complex engineered systems like aircraft, automobiles and ballistic missiles. Tracking problems also arise in actuarial science, biostatistics, cancer research and mathematical finance.

A natural approach for addressing such problems is via the control theory methods of filtering, smoothing and prediction. But to invoke such methods, one needs a proper philosophical foundation, and this has been lacking. The first three sections of this paper endeavour to fill this gap. A consequence is the point of view proposed here, namely, that reliability not be interpreted as a probability. Rather, reliability should be conceptualized as a dynamically evolving propensity in the sense of Pierce and Popper. Whereas propensity is to be taken as an undefined primitive, it manifests as a chance (or frequency) in the sense of de Finetti. The idea of looking at reliability as a propensity also appears in the philosophical writings of Kolmogorov. Furthermore, survivability which quantifies ones uncertainty about a propensity should be the metric of performance that needs to be tracked. The first part of this paper is thus a proposal for a paradigm shift in the manner in which one conceptualizes reliability, and by extension, survival analysis. This message is also germane to other areas of applied probability and statistics, like queueing, inventory and time series analysis.

The second part of this paper is technical. Its purpose is to show how the philosophical material of the first part can be incorporated into a framework that leads to a methodological package. To do so, we focus on the problem which motivated the first part, and develop a mathematical model for describing the evolution of an item’s propensity to survive. The item could be a component, a system or a biological entity. The proposed model is based on the notion of competing risks. Models like this also appear in biostatistics under the label of cure models. Whereas the competing risks scenario is instructive, it is not the only way to describe the phenomenon of growth; its use here is illustrative. All the same, one of its virtues is that it paves the path towards a contribution to the state of the art of filtering by considering the case of censored observations. Even though censoring is the hallmark of survival analysis, it could also arise in time series analysis and control theory, making the development here of a broader and more general appeal.

Article information

Statist. Sci., Volume 31, Number 4 (2016), 521-540.

First available in Project Euclid: 19 January 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Chance competing risks cure models exchangeability filtering censored observations frailty propensity proportional hazards time series analysis


Singpurwalla, Nozer D. Filtering and Tracking Survival Propensity (Reconsidering the Foundations of Reliability). Statist. Sci. 31 (2016), no. 4, 521--540. doi:10.1214/16-STS565.

Export citation


  • Aalen, O. O. (1988). Heterogenity in survival analysis. Stat. Med. 7 1121–1137.
  • Barlow, R. E. and Mendel, M. B. (1992). De Finetti-type representations for life distributions. J. Amer. Statist. Assoc. 87 1116–1122.
  • Barlow, R. E. and Proschan, F. (1965). Mathematical Theory of Reliability. Wiley, New York-London-Sydney. With Contributions by Larry C. Hunter.
  • Barlow, R. E. and Proschan, F. (1975). Statistical Theory of Reliability and Life Testing: Probability Models. Holt, Rinehart and Winston, Inc., New York.
  • Bather, J. A. (1965). Invariant conditional distributions. Ann. Math. Stat. 36 829–846.
  • Block, H. W. and Savits, T. H. (1997). Burn-in. Statist. Sci. 12 1–13.
  • Chen, M.-H., Ibrahim, J. G. and Sinha, D. (1999). A new Bayesian model for survival data with a surviving fraction. J. Amer. Statist. Assoc. 94 909–919.
  • Clayton, D. G. (1991). A Monte Carlo method for Bayesian inference in frailty models. Biometrics 47 467–485.
  • Clayton, D. and Cuzick, J. (1985). Multivariate generalizations of the proportional hazards model. J. Roy. Statist. Soc. Ser. A 148 82–117.
  • Cox, D. R. (1972). Regression models and life-tables. J. R. Stat. Soc. Ser. B. Stat. Methodol. 34 187–220. With discussion by F. Downton, Richard Peto, D. J. Bartholomew, D. V. Lindley, P. W. Glassborow, D. E. Barton, Susannah Howard, B. Benjamin, John J. Gart, L. D. Meshalkin, A. R. Kagan, M. Zelen, R. E. Barlow, Jack Kalbfleisch, R. L. Prentice and Norman Breslow, and a reply by D. R. Cox.
  • de Finetti, B. (1937). La prévision : Ses lois logiques, ses sources subjectives. Ann. Inst. Henri Poincaré 7 1–68.
  • de Groot, M. H. (1988). A Bayesian view of assessing uncertainty and comparing expert opinion. J. Statist. Plann. Inference 20 295–306.
  • Diaconis, P. and Freedman, D. (1987). A dozen de Finetti-style results in search of a theory. Ann. Inst. Henri Poincaré Probab. Stat. 23 397–423.
  • Dubey, S. D. (1966). Compound Pascal distributions. Ann. Inst. Statist. Math. 18 357–365.
  • Escobar, L. A. and Meeker, W. Q. (2006). A review of accelerated test models. Statist. Sci. 21 552–577.
  • Fetzer, J. H. (1981). Scientific Knowledge: Causation, Explanation, and Corroboration. Boston Stud. Philos. Sci. 69. D. Reidel, Dordrecht.
  • Gillies, D. (2000). Varieties of propensity. British J. Philos. Sci. 51 807–835.
  • Good, I. J. (1950). Probability and the Weighing of Evidence. Charles Griffin & Co., Ltd., London.
  • Good, I. J. and Card, W. I. (1971). The diagnostic process with special reference to errors. Methods Inf. Med. 10 176–188.
  • Hamada, M. S., Wilson, A. G., Reese, C. S. and Martz, H. F. (2008). Bayesian Reliability. Springer, New York.
  • Hewitt, E. and Savage, L. J. (1955). Symmetric measures on Cartesian products. Trans. Amer. Math. Soc. 80 470–501.
  • Howson, C. and Urbach, P. (2006). Scientific Reasoning: The Bayesian Approach. Open Court, Chicago, IL.
  • Humphreys, P. (1985). Why propensities cannot be probabilities. Philos. Rev. 94 557–570.
  • Kendall, M. G. (1949). On the reconciliation of theories of probability. Biometrika 36 101–116.
  • Kolmogorov, A. N. (1969). The theory of probability. In Mathematics: Its Content, Methods and Meaning, Vol. 2, Part 3. (A. D. Aleksandrov, A. N. Kolmogorov and M. A. Lavrentav, eds.). MIT Press, Cambridge, MA.
  • Lindley, D. V. (1990). The 1988 Wald memorial lectures: The present position in Bayesian statistics. Statist. Sci. 5 44–89. With comments and a rejoinder by the author.
  • Lindley, D. V. and Singpurwalla, N. D. (1986). Multivariate distributions for the life lengths of components of a system sharing a common environment. J. Appl. Probab. 23 418–431.
  • Lindley, D. V. and Singpurwalla, N. D. (2002). On exchangeable, causal and cascading failures. Statist. Sci. 17 209–219.
  • Martin, O. S. (2012). A Dynamic Competing Risk Model for Filtering Reliability and Tracking Survivability. ProQuest LLC, Ann Arbor, MI. Ph.D. thesis, The George Washington University.
  • Meeker, W. Q. and Escobar, L. A. (1998). Statistical Methods for Reliability Data. Wiley, New York.
  • Meinhold, R. and Singpurwalla, N. D. (1983). Understanding the Kalman filter. Amer. Statist. 37 123–127.
  • Miller, R. W. (1975). Propensity: Popper or peirce? British J. Philos. Sci. 123–132.
  • Popper, K. R. (1957). The propensity interpretation of the calculus of probability, and the quantum theory. In Observation and Interpretation in the Philosophy of Physics (S. Korner, ed.). Dover, Mineola, NY.
  • Popper, K. R. (1959). The propensity interpretation of probability. British J. Philos. Sci. 10 25–42.
  • Singpurwalla, N. D. (1988). Foundational issues in reliability and risk analysis. SIAM Rev. 30 264–282.
  • Singpurwalla, N. D. (1995). Survival in dynamic environments. Statist. Sci. 10 86–113.
  • Singpurwalla, N. D. (2002). Some cracks in the empire of chance (flaws in the foundations of reliability). Int. Stat. Rev. 70 53–65.
  • Singpurwalla, N. D. (2006). Reliability and Risk: A Bayesian Perspective. Wiley, Chichester.
  • Singpurwalla, N. D. (2007). Betting on residual life: The caveats of conditioning. Statist. Probab. Lett. 77 1354–1361.
  • Singpurwalla, N. D. (2010). Comment: “Reliability growth management metrics and statistical methods for discrete-use systems” [MR2789242]. Technometrics 52 396–397.
  • Singpurwalla, N. D. (2016). The indicative and irrealis moods in Bayesian inference. Under review.
  • Singpurwalla, N. D. and Wilson, P. (2004). When can finite testing ensure infinite trustworthiness? J. Iran. Stat. Soc. (JIRSS) 3 1–37.
  • Smith, A. F. M. (1981). On random sequences with centred spherical symmetry. J. R. Stat. Soc. Ser. B. Stat. Methodol. 43 208–209.
  • Spizzichino, F. (2001). Subjective Probability Models for Lifetimes. Monographs on Statistics and Applied Probability 91. Chapman & Hall/CRC, Boca Raton, FL.
  • Tsodikov, A. D., Ibrahim, J. G. and Yakovlev, A. Y. (2003). Estimating cure rates from survival data: An alternative to two-component mixture models. J. Amer. Statist. Assoc. 98 1063–1078.
  • von Mises, R. (1941). On the foundations of probability and statistics. Ann. Math. Stat. 12 191–205.

See also