AI Extracts Broader Health Insights From Routine CT Scans

A new study suggests that CT scans can provide information about a wide range of conditions, including some not typically diagnosed through imaging. Direct comparison with human radiologists is the next step [1].

Getting more from a scan

A CT scan’s goal is usually to answer a concrete clinical question, such as what is causing abdominal pain or whether a suspected tumor is there. However, the images also capture other tissues and organs. The amount of information in a scan greatly exceeds what any radiologist can possibly register and may point to unrelated and yet undiagnosed conditions. A new study from the University of Pennsylvania, published in npj Digital Medicine, describes an AI model that can glean such clues, raise alarms, and assess future risks.

The authors developed Percival, a vision-language model (VLM) trained on CT images and their accompanying radiology reports. The model was trained on more than 400,000 CT image volumes paired with radiology reports from 52,000 participants. These came from the Penn Medicine BioBank and included different anatomical regions, scanner manufacturers, and imaging protocols, with and without contrast.

The model, which uses 86 million parameters, was trained to match images to their corresponding reports. It produces a numerical representation of each scan (an embedding) that encodes patterns it learned from the images.

Signal detected

Principal component analysis showed that the embeddings reflected differences in patients’ age, sex, and body size. This served as a preliminary check that the model captures meaningful patient characteristics. However, associations with blood pressure, pulse, oxygen saturation, and respiratory rate were weak.

The researchers then tested associations between the embeddings and dozens of types of laboratory measurements, taken within 60 days of imaging. They found significant associations involving 35 measurements (corrected for multiple comparisons), including markers related to kidney function, liver function, glucose and lipid metabolism, blood-cell counts, and inflammation.

This suggested that the scans contain broader physiological information. However, when the researchers tried estimating lab values directly, performance was largely weak. Only some measurements showed predictive power, including albumin, hemoglobin, and creatinine.

The researchers then asked whether the embeddings corresponded to diagnoses across multiple organ systems. They found associations with 351 diagnosis categories, spanning cardiovascular, respiratory, cancer, metabolic, and other conditions, documented or diagnosed around the time of the scan. They also found that some image patterns were associated with a greater number of diagnoses recorded after imaging (“future disease burden”).

After establishing these broad associations, the researchers trained disease-specific classifiers to distinguish people with and without recorded diagnoses. They compared Percival with CT-FM, an AI model trained using images without paired reports, which Percival outperformed. The results were the strongest for atrial fibrillation, coronary atherosclerosis, heart failure, and pancreatic cancer. For some of those conditions, such as atrial fibrillation, primary diagnosis does not ordinarily rely on CT.

Future disease prediction

The ultimate test was predicting later diagnoses. Percival competed with an image-only AI model, as well as with predictions that used organ measurements or age and sex, outperforming all of them. Top results were achieved for heart failure, cirrhosis, and acute kidney failure.

“Percival can help identify unrecognized disease, predict future progression, and anticipate major cardiovascular and liver complications,” said co-author Charles Kahn, a professor of radiology. “Because many patients already have substantial heart or metabolic disease when they enter specialty care, routine imaging may provide an opportunity for earlier recognition and intervention.”

However, a later recorded diagnosis does not necessarily mean that a disease developed after imaging. The researchers did not exclude diagnoses made shortly afterwards. Some predictions may therefore relate to an existing condition that had not yet been formally documented. It is a valuable capability but different from forecasting disease years before onset.

“Our teams have developed something that learns broadly from imaging data and uncovers connections between scans, lab values, people’s physiology, and clinical outcomes to potentially give more accurate assessments of health or even ring alarm bells early,” said senior author Walter Witschey, an associate professor of radiology and the associate vice chair of research IT. “This is work that points toward a future in which medical imaging becomes a foundation for more personalized and proactive health care.”

The study suggests that there is potential value in vision-language models that can combine several types of data from the patient’s health records. However, no direct comparison with human radiologists was made, so the study does not establish that Percival is better at diagnosing the conditions a CT scan was ordered to investigate.

Literature

[1] Beeche, C. A., Kim, J., Tavolinejad, H., Zhao, B., Dong, J., Sharma, R., … & Penn Medicine Biobank. (2026). Generalizable CT vision-language modeling for population health and disease risk. npj Digital Medicine.