Artificial Intelligence in healthcare is racist Posted on : Nov 02 - 2020

Can broader datasets help developers avoid accidentally perpetuating deep-rooted biases in vital institutions like healthcare and education?

AI in healthcare has a bias problem. Last year, it came to light that six algorithms used on an estimated 60-100 million patients nationwide were prioritizing care coordination for white patients over black patients for the same level of illness.

The reason? The algorithm was trained on costs in insurance claims data, predicting which patients would be expensive in the future based on who was expensive in the past. Historically, less is spent on black patients than white patients, so the algorithm ended up perpetuating existing bias in healthcare.

Therein lies the danger of using narrow datasets in Artificial Intelligence: If the data is biased, the AI will be biased. That doesn't mean we should (or, now that the genie is out of the bottle, can) abandon AI. Which leads to an obvious question: Can using broader datasets, including socioeconomic data, reduce the influence of bias in clinical AI and correct systemic bias that persists in vital institutions like healthcare, education, and law enforcement?

Dr. John Frownfelter, Chief Medical Information Officer at Jvion, is one of the people advancing the broader dataset approach. Jvion's AI analyzes over 4,500 factors per patient so bias in any one dataset doesn't compromise the integrity of the AI's output. Jvion's AI also actively counters existing bias by flagging socioeconomic barriers to care that drive bias in the first place.

I caught up with Dr. Frownfelter to understand how developers and organizations can get the inevitable shift to AI to drive healthcare delivery right. 

GN: How pervasive is AI becoming in fields that impact patient health and outcomes? Can you give us a sense of the trajectory of adoption over the past few years?

Dr. Frownfelter: The market for AI in healthcare is growing exponentially, from $600M to $6B in the last few years. Gartner predicts that by 2021, 75% of provider organizations will have invested in AI to either improve operational performance or clinical outcomes. Our own AI at Jvion is now in use at over 300 hospitals and 40 health systems, with a database encompassing 30M+ patients. So far, radiology is furthest ahead for AI in healthcare, where machine learning models are being used to detect malignancies and other abnormalities in MRIs, X-rays and other scans.

It's getting to the point now where AI will be critical to hospitals' survival in the near future, particularly as the pandemic bleeds providers of revenue, and more providers shift to value-based care models that tie their financial outcomes to improving patient outcomes. With clinical AI, it becomes possible to leverage patient data to make more informed clinical decisions that ultimately improve patient outcomes. View More