Statistical Science

Regression Theory for Categorical Time Series

Konstantinos Fokianos and Benjamin Kedem

Full-text: Open access


Categorical---or qualitative---time series data with random time-dependent covariates are frequently encountered in diverse applications as the list of examples shows. As with "ordinary'' time series, the data analyst is faced with the same problems of modeling, estimation, model checking, diagnostics and prediction. The present work shows that these questions can be attacked by means of regression theory for categorical time series whose foundation is based on generalized linear models and partial likelihood inference. A variety of models are provided to illustrate the selection of the link function and recent large sample results are reviewed. The theory is developed without resorting to the Markov assumption and to the notion of stationarity. Moreover, regression methods for categorical time series allow for parsimonious modeling and incorporation of random time-dependent covariates as opposed to other procedures. In particular, nominal and ordinal time series are analyzed and compared empirically to Markov chains and mixture transition distribution models.

Article information

Statist. Sci., Volume 18, Issue 3 (2003), 357-376.

First available in Project Euclid: 6 February 2004

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Random time-dependent covariates partial likelihood martingale multinomial logits proportional odds link function deviance residuals Markov chain mixture transition distribution model


Fokianos, Konstantinos; Kedem, Benjamin. Regression Theory for Categorical Time Series. Statist. Sci. 18 (2003), no. 3, 357--376. doi:10.1214/ss/1076102425.

Export citation


  • Adke, S. R. and Deshmukh, S. R. (1988). Limit distributions of a high order Markov chain. J. Roy. Statist. Soc. Ser. B 50 105--108.
  • Agresti, A. (1990). Categorical Data Analysis. Wiley, New York.
  • Agresti, A. (1999). Modeling ordered categorical data: Recent advances and future challenges. Statistics in Medicine 18 2191--2207.
  • Al-Osh, M. A. and Alzaid, A. A. (1987). First-order integer-valued autoregressive (INAR($1$)) process. J. Time Ser. Anal. 8 261--275.
  • Alzaid, A. A. and Al-Osh, M. A. (1990). An integer-valued $p$th-order autoregressive structure (INAR($p$)) process. J. Appl. Probab. 27 314--324.
  • Ashby, M., Neuhaus, J., Hauck, W., Bacchetti, P., Heilbron, D., Jewell, N., Segal, M. and Fusaro, R. (1992). An annotated bibliography of methods for analyzing correlated categorical data. Statistics in Medicine 11 67--99.
  • Azzalini, A. (1983). Maximum likelihood estimation of order $m$ for stationary stochastic process. Biometrika 70 381--387.
  • Basawa, I. V. and Prakasa Rao, B. L. S. (1980). Statistical Inference for Stochastic Processes. Academic Press, London.
  • Berchtold, A. (1999). The double chain Markov model. Comm. Statist. Theory Methods 28 2569--2589.
  • Berchtold, A. (2001). Estimation in the mixture transition distribution model. J. Time Ser. Anal. 22 379--397.
  • Berchtold, A. and Raftery, A. E. (1999). The mixture transition distribution (mtd) model for high-order Markov chains and non-Gaussian time series. Technical Report 360, Dept. Statist., Univ. Washington, Seattle.
  • Billingsley, P. (1961). Statistical Inference for Markov Processes. Univ. Chicago Press.
  • Box, G. P., Jenkins, G. M. and Reinsel, G. C. (1994). Time Series Analysis, Forecasting and Control, 3rd. ed. Prentice-Hall, Englewood Cliffs, NJ.
  • Brillinger, D. R. (1996). An analysis of an ordinal-valued time series. In Athens Conference on Applied Probability and Time Series, II: Time Series Analysis. Lecture Notes in Statist. 115 73--87. Springer, New York.
  • Brillinger, D. R., Morettin, P. A., Irizarry, R. A. and Chiann, C. (2000). Some wavelet-based analyses of Markov chain data. Signal Processing 80 1607--1627.
  • Bühlmann, P. and Wyner, A. J. (1999). Variable length Markov chains. Ann. Statist. 27 480--513.
  • Clayton, D. G. (1992). Repeated ordinal measurements: A generalized estimating equation approach. Technical report, Medical Research Council Biostatistics Unit, Cambridge, UK.
  • Cox, D. R. (1975). Partial likelihood. Biometrika 62 269--276.
  • Diggle, P. J., Liang, K.-Y. and Zeger, S. L. (1994). Analysis of Longitudinal Data. Oxford Univ. Press, New York.
  • Fahrmeir, L. (1987). Asymptotic testing theory for generalized linear models. Statistics 18 65--76.
  • Fahrmeir, L. and Kaufmann, H. (1987). Regression models for nonstationary categorical time series. J. Time Ser. Anal. 8 147--160.
  • Fahrmeir, L. and Pritscher, L. (1996). Regression analysis of forest damage by marginal models for correlated ordinal responses. Environ. Ecol. Stat. 3 257--268.
  • Fahrmeir, L. and Tutz, G. (2001). Multivariate Statistical Modelling Based on Generalized Linear Models, 2nd ed. Springer, New York.
  • Fokianos, K. (2002). Power divergence family of tests for categorical time series models. Ann. Inst. Statist. Math. 54 543--564.
  • Fokianos, K. and Kedem, B. (1998). Prediction and classification of non-stationary categorical time series. J. Multivariate Anal. 67 277--296.
  • Fokianos, K., Kedem, B. and Short, D. (1996). Predicting precipitation level. J. Geophys. Res. D: Atmospheres 101 $26,473$--$26,477$.
  • Guttorp, P. (1995). Stochastic Modelling of Scientific Data. Chapman and Hall, London.
  • Heagerty, P. J. and Zeger, S. L. (1996). Marginal regression models for clustered ordinal measurements. J. Amer. Statist. Assoc. 91 1024--1036.
  • Heagerty, P. J. and Zeger, S. L. (1998). Lorelogram: A regression approach to exploring dependence in longitudinal categorical responses. J. Amer. Statist. Assoc. 93 150--162.
  • Jacobs, P. A. and Lewis, P. A. W. (1978a). Discrete time series generated by mixtures. I. Correlational and runs properties. J. Roy. Statist. Soc. Ser. B 40 94--105.
  • Jacobs, P. A. and Lewis, P. A. W. (1978b). Discrete time series generated by mixtures. II. Asymptotic properties. J. Roy. Statist. Soc. Ser. B 40 222--228.
  • Johnson, V. E. and Albert, J. H. (1999). Ordinal Data Modeling. Springer, New York.
  • Karlin, S. and Taylor, H. M. (1975). A First Course in Stochastic Processes, 2nd ed. Academic Press, New York.
  • Kaufmann, H. (1987). Regression models for nonstationary categorical time series: Asymptotic estimation theory. Ann. Statist. 15 79--98.
  • Kedem, B. (1980). Binary Time Series. Dekker, New York.
  • Kedem, B. (1994). Time Series Analysis by Higher Order Crossings. IEEE Press, New York.
  • Kedem, B. and Kozintsev, B. (2000). Graphical bootstrap. In Proc. Section on Statistics and the Environment 30--32. Amer. Statist. Assoc., Alexandria, VA.
  • Keenan, D. M. (1982). A time series analysis of binary data. J. Amer. Statist. Assoc. 77 816--821.
  • Kosorok, M. R. and Chao, W.-H. (1996). The analysis of longitudinal ordinal response data in continuous time. J. Amer. Statist. Assoc. 91 807--817.
  • Kozintsev, B. and Kedem, B. (2000). Generation of ``similar'' images from a given discrete image. J. Comput. Graph. Statist. 9 286--302.
  • Le, N. D., Martin, R. D. and Raftery, A. E. (1996). Modelling flat stretches, bursts, and outliers in time series using mixture transition distribution models. J. Amer. Statist. Assoc. 91 1504--1515.
  • Luce, R. D. (1959). Individual Choice Behavior. Wiley, New York.
  • MacDonald, I. L. and Zucchini, W. (1997). Hidden Markov and Other Models for Discrete-Valued Time Series. Chapman and Hall, London.
  • McCullagh, P. (1980). Regression models for ordinal data (with discussion). J. Roy. Statist. Soc. Ser. B 42 109--142.
  • McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd ed. Chapman and Hall, London.
  • McFadden, D. (1973). Conditional logit analysis of qualitative choice behavior. In Frontiers in Econometrics (P. Zarembka, ed.) 105--142. Academic Press, New York.
  • McKenzie, E. (1985). Some simple models for discrete variate time series. Water Res. Bull. 21 645--650.
  • McKenzie, E. (1986). Autoregressive moving-average processes with negative-binomial and geometric marginal distributions. Adv. in Appl. Probab. 18 679--705.
  • McKenzie, E. (1988). Some ARMA models for dependent sequences of Poisson counts. Adv. in Appl. Probab. 20 822--835.
  • Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.
  • Miller, M. E., Davis, C. S. and Landis, J. R. (1993). The analysis of longitudinal polytomous data: Generalized estimated equations and connections with weighted least squares. Biometrics 49 1033--1044.
  • Molenberghs, G. and Lesaffre, E. (1999). Marginal modelling of multivariate categorical data. Statistics in Medicine 18 2237--2255.
  • Nason, G. P., Sapatinas, T. and Sawczenko, A. (2001). Wavelet packet modeling of infant sleep state using heart rate data. Sankhyā Ser. B 63 199--217.
  • Pegram, G. G. S. (1980). An autoregressive model for multilag Markov chains. J. Appl. Probab. 17 350--362.
  • Pendergast, J. F., Gange, S. J., Lindstrom, M. J., Newton, M. A., Palta, M. and Fisher, M. R. (1996). A survey of methods for analyzing clustered binary response data. Internat. Statist. Rev. 64 89--118.
  • Priestley, M. B. (1981). Spectral Analysis and Time Series. Academic Press, London.
  • Pruscha, H. (1993). Categorical time series with a recursive scheme and with covariates. Statistics 24 43--57.
  • Raftery, A. E. (1985a). A model for high-order Markov chains. J. Roy. Statist. Soc. Ser. B 47 528--539.
  • Raftery, A. E. (1985b). A new model for discrete-valued time series: Autocorrelations and extensions. Rassegna di Metodi Statistici ed Applicazioni 3--4 149--162.
  • Raftery, A. E. and Banfield, J. D. (1991). Stopping the Gibbs sampler, the use of morphology and other issues in spatial statistics. Ann. Inst. Statist. Math. 43 32--43.
  • Raftery, A. E. and Tavaré, S. (1994). Estimation and modelling repeated patterns in high order Markov chains with the mixture transition distribution model. Appl. Statist. 43 179--199.
  • Read, T. R. C. and Cressie, N. A. C. (1988). Goodness-of-Fit Statistics for Discrete Multivariate Data. Springer, New York.
  • Shumway, R. H. and Stoffer, D. S. (2000). Time Series Analysis and Its Applications. Springer, New York.
  • Slud, E. V. and Kedem, B. (1994). Partial likelihood analysis of logistic regression and autoregression. Statist. Sinica 4 89--106.
  • Snell, E. J. (1964). A scaling procedure for ordered categorical data. Biometrics 20 592--607.
  • Stoffer, D. S., Tyler, D. E. and McDougall, A. J. (1993). Spectral analysis of categorical time series: Scaling and the spectral envelope. Biometrika 80 611--622.
  • Stoffer, D. S., Tyler, D. E. and Wendt, D. A. (2000). The spectral envelope and its applications. Statist. Sci. 15 224--253.
  • Stram, D. O., Wei, L. J. and Ware, J. H. (1988). Analysis of repeated ordered categorical outcomes with possibly missing observations and time-dependent covariates. J. Amer. Statist. Assoc. 83 631--637.
  • Sutradhar, B. C. and Kovacevic, M. (2000). Analysing ordinal longitudinal survey data: Generalised estimating equations approach. Biometrika 87 837--848.
  • Waterman, M. S. (1995). Introduction to Computational Biology: Maps, Sequences and Genomes. Chapman and Hall, New York.
  • Williamson, J. M., Kim, K. M. and Lipsitz, S. R. (1995). Analyzing bivariate ordinal data using a global odds ratio. J. Amer. Statist. Assoc. 90 1432--1437.
  • Wong, C. S. and Li, W. K. (2000). On a mixture autoregressive model. J. R. Stat. Soc. Ser. B Stat. Methodol. 62 95--115.
  • Wong, W. H. (1986). Theory of partial likelihood. Ann. Statist. 14 88--123.