March 2024 Composite scores for transplant center evaluation: A new individualized empirical null method
Nicholas Hartman, Joseph M. Messana, Jian Kang, Abhijit S. Naik, Tempie H. Shearon, Kevin He
Author Affiliations +
Ann. Appl. Stat. 18(1): 729-748 (March 2024). DOI: 10.1214/23-AOAS1809


Risk-adjusted quality measures are used to evaluate healthcare providers with respect to national norms while controlling for factors beyond their control. Existing healthcare provider profiling approaches typically assume that the between-provider variation in these measures is entirely due to meaningful differences in quality of care. However, in practice, much of the between-provider variation will be due to trivial fluctuations in healthcare quality or unobservable confounding risk factors. If these additional sources of variation are not accounted for, conventional methods will disproportionately identify larger providers as outliers, even though their departures from the national norms may not be “extreme” or clinically meaningful. Motivated by efforts to evaluate the quality of care provided by transplant centers, we develop a composite evaluation score based on a novel individualized empirical null method, which robustly accounts for overdispersion due to unobserved risk factors, models the marginal variance of standardized scores as a function of the effective sample size, and only requires the use of publicly-available center-level statistics. The evaluations of United States kidney transplant centers based on the proposed composite score are substantially different from those based on conventional methods. Simulations show that the proposed empirical null approach more accurately classifies centers in terms of quality of care, compared to existing methods.

Funding Statement

This work was partially supported by the Centers for Medicare and Medicaid Services under Contract 75FCMC21C0035 and the National Institute of Diabetes and Digestive and Kidney Diseases under Grant R01DK129539. The statements contained in this article are solely those of the authors and do not necessarily reflect the views or policies of the Centers for Medicare and Medicaid Services or the National Institute of Diabetes and Digestive and Kidney Diseases.


The authors would like to thank Dr. John D. Kalbfleisch for his helpful comments. The data that support the findings of this paper are publicly available from the Scientific Registry of Transplant Recipients at


Download Citation

Nicholas Hartman. Joseph M. Messana. Jian Kang. Abhijit S. Naik. Tempie H. Shearon. Kevin He. "Composite scores for transplant center evaluation: A new individualized empirical null method." Ann. Appl. Stat. 18 (1) 729 - 748, March 2024.


Received: 1 August 2022; Revised: 1 June 2023; Published: March 2024
First available in Project Euclid: 31 January 2024

Digital Object Identifier: 10.1214/23-AOAS1809

Keywords: empirical null , end-stage renal disease , provider profiling , unmeasured confounders

Rights: Copyright © 2024 Institute of Mathematical Statistics


This article is only available to subscribers.
It is not available for individual sale.

Vol.18 • No. 1 • March 2024
Back to Top