IIIT-H study caution doctors against relying entirely on AI for medical diagnosis

Doctors should exercise caution when using artificial intelligence (AI) tools to interpret medical images, as areas highlighted by AI models may not always align with those identified by radiologists, according to a study by researchers at the International Institute of Information Technology, Hyderabad (IIIT-H).

The study, conducted by the Language Technologies Research Centre (LTRC) at IIIT-H, examined four vision-language models for analysing chest X-rays and found discrepancies between the regions highlighted by the models and those identified by radiologists.

The findings underscore the need for doctors to independently assess AI-generated outputs rather than relying on them entirely for diagnosis.

The research team, led by Parameswari Krishnamurthy, sought to determine whether AI-generated heatmaps accurately represented the regions of an image that radiologists would identify as disease-related.

However, the researchers questioned whether these visual indicators necessarily reflected the models’ ability to identify the actual location of a disease.

“We essentially wanted to examine whether the heatmaps created by vision-language models actually correspond to where radiologists, who look at the image, would say the disease lies,” said Syed Faizan, principal investigator of the study titled “How Well Do Chest X-Ray VLM Attention Overlays Match Radiologist Boxes? A Cross-Model Audit and Radiologist Reader.”

The team evaluated four models, MAIRA-2, MedGemma-4B, LLaVA-Med-1.5, and LLaVA-1.5, against thousands of publicly available chest X-rays. Two radiologists also participated in the study to assess them, allowing the researchers to compare AI-generated highlights with human assessments.