The Annals of Applied Statistics

Biomarker assessment and combination with differential covariate effects and an unknown gold standard, with an application to Alzheimer’s disease

Zheyu Wang and Xiao-Hua Zhou

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


The continued efforts to evaluate biomarkers’ predictive abilities and identify optimal biomarker combinations are often challenged by the absence of a gold standard, that is, the true disease status. Current methods that address this issue are mostly developed for binary or ordinal diagnostic tests, which do not fully utilize information provided by continuous biomarkers, or require strong parametric assumptions. Moreover, limited methods exist to allow for the inclusion of covariates—despite their crucial role in facilitating the accurate evaluation of biomarkers. In this paper, we proposed a latent profile approach to evaluating diagnostic accuracy of biomarkers without a gold standard. The method allows for flexible biomarker distributions and incorporation of previous knowledge about risk factors while simultaneously permitting researchers to model paticipants’ characteristics that putatively affect biomarker levels, and therefore provides information needed to develop more personalized diagnostic procedures. Additionally, the proposed method presents a potential strategy for biomarker combination when gold standard information is unavailable, as it derives a composite risk score for the underlying disease status. The method is applied to evaluate different cerebral spinal fluid (CSF) biomarkers for Alzheimer’s disease (AD) detection. The results show that CSF biomarkers hold significant potential for facilitating early AD detection and for continuous disease monitoring. Furthermore, they call attention to biomarker variability in subgroups and reexamination of CSF biomarker distributions. Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database.

Article information

Ann. Appl. Stat., Volume 12, Number 2 (2018), 1204-1227.

Received: May 2016
Revised: July 2017
First available in Project Euclid: 28 July 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Diagnostic accuracy latent profile model finite mixture models differential covariate effect identifiability Alzheimer’s disease


Wang, Zheyu; Zhou, Xiao-Hua. Biomarker assessment and combination with differential covariate effects and an unknown gold standard, with an application to Alzheimer’s disease. Ann. Appl. Stat. 12 (2018), no. 2, 1204--1227. doi:10.1214/17-AOAS1085.

Export citation


  • Albert, P. S. and Dodd, L. E. (2004). A cautionary note on the robustness of latent class models for estimating diagnostic error without a gold standard. Biometrics 60 427–435.
  • Albert, P. S., McShane, L. M. and Shih, J. H. (2001). Latent class modelling approaches for assessing diagnostic error without a gold standard: With applications to p53 immunohistochemical assays in bladder tumors. Biometrics 57 610–619.
  • Bandeen-Roche, K., Miglioretti, D. L., Zeger, S. L. and Rathouz, P. J. (1997). Latent variable regression for multiple discrete outcomes. J. Amer. Statist. Assoc. 92 1375–1386.
  • Bateman, R. J., Xiong, C., Benzinger, T. L. S., Fagan, A. M., Goate, A., Fox, N. C., Marcus, D. S., Cairns, N. J., Xie, X., Blazey, T. M., Holtzman, D. M., Santacruz, A., Buckles, V., Oliver, A., Moulder, K., Aisen, P. S., Ghetti, B., Klunk, W. E., McDade, E., Martins, R. N., Masters, C. L., Mayeux, R., Ringman, J. M., Rossor, M. N., Schofield, P. R., Sperling, R. A., Salloway, S., Morris, J. C. and Dominantly Inherited Alzheimer Network (2012). Clinical and biomarker changes in dominantly inherited Alzheimer’s disease. N. Engl. J. Med. 367 795–804.
  • Benaglia, T., Chauveau, D. and Hunter, D. R. (2009). An EM-like algorithm for semi- and nonparametric estimation in multivariate mixtures. J. Comput. Graph. Statist. 18 505–526.
  • Box, G. E. P. and Cox, D. R. (1964). An analysis of transformations. J. Roy. Statist. Soc. Ser. B 26 211–252.
  • Branscum, A. J., Johnson, W. O., Hanson, T. E. and Gardner, I. A. (2008). Bayesian semiparametric ROC curve estimation and disease diagnosis. Stat. Med. 27 2474–2496.
  • Branscum, A. J., Johnson, W. O., Hanson, T. E. and Baron, A. T. (2015). Flexible regression models for ROC and risk analysis, with or without a gold standard. Stat. Med. 34 3997–4015.
  • Cheng, R. C. H. and Traylor, L. (1995). Non-regular maximum likelihood problems. J. Roy. Statist. Soc. Ser. B 57 3–44.
  • Collins, J. and Huynh, M. (2014). Estimation of diagnostic test accuracy without full verification: A review of latent class methods. Stat. Med. 33 4141–4169.
  • Cook, R. J., Ng, E. T. M. and Meade, M. O. (2000). Estimation of operating characteristics for dependent diagnostic tests based on latent Markov models. Biometrics 56 1109–1117.
  • Efron, B. (1981). Nonparametric estimates of standard error: The jackknife, the bootstrap and other methods. Biometrika 68 589–599.
  • Efron, B. (1987). Better bootstrap confidence intervals. J. Amer. Statist. Assoc. 82 171–200.
  • Goodman, L. A. (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika 61 215–231.
  • Hebert, L. E., Weuve, J., Scherr, P. A. and Evans, D. A. (2013). Alzheimer disease in the United States (2010–2050) estimated using the 2010 census. Neurology 80 1778–1783.
  • Huang, G.-H. and Bandeen-Roche, K. (2004). Building an identifiable latent class model with covariate effects on underlying and measured variables. Psychometrika 69 5–32.
  • Hui, S. L. and Walter, S. D. (1980). Estimating the error rates of diagnostic tests. Biometrics 36 167–171.
  • Jack, C. R. Jr., Knopman, D. S., Jagust, W. J., Shaw, L. M., Aisen, P. S., Weiner, M. W., Petersen, R. C. and Trojanowski, J. Q. (2010). Hypothetical model of dynamic biomarkers of the Alzheimer’s pathological cascade. Lancet Neurol. 9 119–128.
  • Janes, H. and Pepe, M. S. (2009). Adjusting for covariate effects on classification accuracy using the covariate-adjusted receiver operating characteristic curve. Biometrika 96 371–382.
  • Jones, G., Johnson, W. O., Vink, W. D. and French, N. (2012). A framework for the joint modeling of longitudinal diagnostic outcome data and latent infection status: Application to investigating the temporal relationship between infection and disease. Biometrics 68 371–379.
  • Lazarsfeld, P. F. and Henry, N. W. (1968). Latent Structure Analysis. Houghton Mifflin, New York.
  • Lehmann, E. L. and Casella, G. (1998). Theory of Point Estimation, 2nd ed. Springer, New York.
  • McHugh, R. B. (1956). Efficient estimation and local identification in latent class analysis. Psychometrika 21 331–347.
  • McLachlan, G. and Peel, D. (2004). Finite Mixture Models. Wiley-Interscience, New York.
  • Pepe, M. S. (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction. Oxford Statistical Science Series 28. Oxford Univ. Press, Oxford.
  • Pfeiffer, R. M., Carroll, R. J., Wheeler, W., Whitby, D. and Mbulaiteye, S. (2008). Combining assays for estimating prevalence of human herpesvirus 8 infection using multivariate mixture models. Biostatistics 9 137–151.
  • Qu, Y., Tan, M. and Kutner, M. H. (1996). Random effects models in latent class analysis for evaluating accuracy of diagnostic tests. Biometrics 52 797–810.
  • Redner, R. A. and Walker, H. F. (1984). Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev. 26 195–239.
  • Selkoe, D. J. (1991). The molecular pathology of Alzheimer’s disease. Neuron 6 487–498.
  • Storandt, M., Head, D., Fagan, A. M., Holtzman, D. M. and Morris, J. C. (2012). Toward a multifactorial model of Alzheimer disease. Neurobiol. Aging 33 2262–2271.
  • van Smeden, M., Naaktgeboren, C. A., Reitsma, J. B., Moons, K. G. and de Groot, J. A. (2013). Latent class models in diagnostic studies when there is no reference standard—A systematic review. Am. J. Epidemiol. 179 423–431.
  • Wang, Z. (2013). Latent Class and Latent Profile Analysis in Medical Diagnosis and Prognosis. Ph.D. thesis, University of Washington.
  • Wang, Z. and Zhou, X.-H. (2012). Random effects models for assessing diagnostic accuracy of traditional Chinese doctors in absence of a gold standard. Stat. Med. 31 661–671.
  • Wang, Z. and Zhou, X.-H. (2014). Nonparametric identifiability of finite mixture models with covariates for estimating error rate without a gold standard. UW Biostatistics Working Paper Series. Working Paper 403.
  • Wang, Z., Zhou, X.-H. and Wang, M. (2011). Evaluation of diagnostic accuracy in detecting ordered symptom statuses without a gold standard. Biostatistics 12 567–581.
  • Wu, Z., Deloria-Knoll, M., Hammitt, L. L. and Zeger, S. L. (2016). Partially latent class models for case-control studies of childhood pneumonia aetiology. J. R. Stat. Soc. Ser. C. Appl. Stat. 65 97–114.
  • Zhou, X.-H., Castelluccio, P. and Zhou, C. (2005). Nonparametric estimation of ROC curves in the absence of a gold standard. Biometrics 61 600–609.