ICUs Overpredict Risk of Death for Black, Hispanic Patients, Study Finds

Research & Discovery

ICUs Overpredict Risk of Death for Black, Hispanic Patients, Study Finds

Three scoring systems used in hospital intensive care units to predict risk of death overestimate it for Hispanic and African Americans, a new study finds. This could potentially affect the care patients receive when resources are scarce, as has occurred during the COVID-19 pandemic.

The widely used scoring systems were “imperfect” for all patients and tended to overpredict the risk of death in general, the study found. But this was particularly true for Black and Hispanic patients, prompting the researchers to urge “extreme caution” in using the results for individual patient care decisions.

“Our finding is a bit inconvenient for those who use these systems for risk adjustment,” said researcher Dr. David J. Stone, of �鶹�ƽ�� Departments of Anesthesiology and Neurosurgery and the Center for Advanced Medical Analytics. “But the data were quite clear, and our major take-home point is that these supposedly objective tools should not be used to make clinical care or triage decisions for individual patients.”

Intensive Care Scoring Systems

Stone and colleagues evaluated three different severity scoring systems used in hospital ICUs: Acute Physiology and Chronic Health Evaluation Iva, or APACHE Iva; Oxford Acute Severity of Illness Score, or OASIS; and Sequential Organ Failure Assessment, or SOFA.

To determine if the systems were susceptible to ethnicity-based bias, the researchers analyzed patient scoring information from two large ICU databases. These databases include de-identified information on more than 166,000 ICU patients in the United States between 2001 and 2015.

Discovery and Innovation: Daily research. Life-changing results.

The authors found that all the scores demonstrate a degree of ethnic bias in their standardized mortality-ratio calibrations in that there was a consistent pattern of overprediction of mortality for African American and Hispanic patients (compared to Asian and white patients). If these scores were used for purposes of triage for individual patients, African American and Hispanic patients could be unfairly denied appropriate access to ICU beds, ventilators and other such resources.

“Extreme care must be taken in the application of current scoring systems for triage decisions in individual patients,” they write, “if they are to be used at all for these purposes in their present states.”

�鶹�ƽ�� Call to Action

The authors go on to call for improved scoring systems that are more useful for making treatment decisions for individual patients. These systems, they say, need to better reflect the communities ICUs serve and the patients they treat. “Our detection of inadvertent, but undeniable, bias in severity scores would seem to indicate that it is time to develop scoring systems that are more precise than the current one-size-fits-all systems,” they write. “Incorporating precise socioeconomic and geographical parameters, along with a set of specific biomarkers for a given disease, into future prediction models might make such models less biased and more robust.”

In addition to the potential effect on individual patients, the scoring systems may be giving a flawed impression of how well ICUs are serving their patients in general, the researchers say. An ICU with a predominantly Black patient population, for example, could appear to be outperforming expectations if fewer patients die than predicted. This could ultimately feed into incorrect assumptions about health disparities and undermine medical research.

“This was not a case of bias in terms of the designs of the scoring systems. We do not know exactly why these findings occurred, but suspect they are a result of pre-existing socio-economic and long-term health care issues that are not taken into account by the score input values,” Stone said. “We don’t currently have the pre-admission data to analyze this hypothesis, but as medicine becomes increasingly digitized, such data will become increasingly available.

“The lesson is, don’t assume that tools like these are 100% objective and complete, because they only capture one small snapshot of an evolving and dynamic clinical and human situation.”

Findings Published

The researchers have in The Lancet Digital Health. The research team consisted of Rahuldeb Sarkar of King’s College, London: Christopher Martin of UCL Institute for Health Informatics; Heather Mattie of Harvard University; Dr. Judy Wawira Gichoya of Emory University; Stone; and Dr. Leo Anthony Celi of the Massachusetts Institute of Technology and Harvard Medical School.

Sarkar disclosed he has received writing fees for health care reports from Crystallise UK. Martin is a director for Crystallise UK.

The work was supported by the National Institutes of Health’s National Institute of Biomedical Imaging and Bioengineering, grant R01 EB017205.

To keep up with the latest medical research news from UVA, subscribe to the blog.