A recent study co-authored by seven Feinberg School of Medicine researchers found that artificial intelligence is more capable than physicians in generating comprehensive oncologic pathology, or cancer diagnosis, reports.
Published on April 8, the study concluded that reports generated by six open-source large language models scored higher in objective metrics compared to physician pathology summaries.
Llama 3.0, Llama 3.1, Llama 3.2, Mistral, Gemma and DeepSeek-R1 were the AI models tested.
Feinberg Prof. Dr. Mohamed Abazeed, one of the co-authors of the study, said he was interested in finding solutions to clinical problems by addressing “workflow deficiencies,” such as human attention to detail.
“As this capability in terms of compute, and in the cost of compute comes down, and the ability to train these really sophisticated models on historical data came into focus, we decided to start filling the gaps,” Abazeed said.
Human attention spans are becoming increasingly limited, and more people are overwhelmed with information and tend to disregard some details, Abazeed said.
Oncology patients tend to have a long treatment history, which means a physician may have to review years of paperwork when some only have about half an hour for patient consultation, said Yirong Liu, the study’s first author and a fifth-year resident in radiation oncology.
Liu said LLMs can help “relieve the burden” of reviewing all of the paperwork.
“The beautiful thing we see in our study is the model can pick up the change of the genetic alteration along the treatment cause, which is quite amazing,” Liu said. “It would take us, as clinician(s), a lot (of) effort to pick up those.”
Troy Teo, a co-author of the study and radiation oncology instructor, said that AI use is inevitable because of how efficient the technology has become.
Still, Teo said hesitance toward using AI in healthcare is reasonable and LLM tools should be implemented in phases.
“It’s logical to be skeptical when new things are being introduced, and that’s why we needed all this benchmark and testing,” Teo said.
Since the study demonstrated that AI models are capable of generating comprehensive reports, Teo said the next step is figuring out how to integrate AI into clinical settings.
Legal and ethical considerations, such as who should be held accountable for mistakes made by AI models, should be factored into their implementation, Teo said.
Liu said he views AI as a tool providing extra information, but does not think it can replace the human aspect of interpersonal relationships in the medical industry.
“You need to make the patient feel comfortable, to trust in you,” Liu said. “The patient prefers to see a human being in person to move forward with the treatment.”
Email: [email protected]
Related Stories:
— Northwestern faculty, students look ahead to the new AI major
— McCormick launches new master’s program in artificial intelligence
