Autoregressive Report Generation for Cardiac Magnetic Resonance Imaging (RF_SA_470)

Saturday, February 1, 2025

11:00 AM – 11:10 AM East Coast USA Time

Location: Blue Room PreFunction

Presenting Author(s)

MN

Makiya Nakashima, MSc

Research Data Scientist II
Cleveland Clinic

Primary Author(s)

MN

Makiya Nakashima, MSc

Research Data Scientist II
Cleveland Clinic

Co-Author(s)

PC

Po-Hao Chen, MD, MBA

Staff
Cleveland Clinic
MB

Michael A. Bolen, MD

Staff Radiologist, Co-Section chief of CVI
Cleveland Clinic
RG

Richard Grimm

Staff
Cleveland Clinic
WT

Wilson Tang, MD

Cardiologist
Cleveland Clinic
Christopher Nguyen, PhD, FSCMR, FACC

Director, Cardiovascular Innovation Research Center
Cleveland Clinic
Deborah Kwon, MD, FSCMR

Director of Cardiac MRI
Cleveland Clinic
DC

David Chen, PhD

Director of Artificial Intelligence
Cleveland Clinic

Background: Producing radiology reports is time consuming task, a primary cause of radiologist burnout, and a factor which limits patient access to advanced diagnostic services. Generative artificial intelligence is increasingly being used to create captions for images, with specific applications in healthcare for automating report generation. However, doing so for cardiac magnetic resonance imaging (CMR) is challenging given the complex nature of exam which incorporates hundreds of images representing cardiac morphology, function, viability, and more. In this study, we propose CMR-Transformer-based AutoRegressive GEnerator for Text (CMR-TARGET), a vision transformer architecture for CMR report generation.

Methods: Consider CMR studies and the associated impressions as {X_n, Y_n}, where the impressions are sequence of tokens Y=(y₁,…,y_k ) for each n ∈ K, where K is the number of tokens in an impression. The images of the CMR studies are processed through a video-transformer resulting in an embedding vl and the text generator is trained using cross-entropy loss for next-word prediction (Figure 1). Visual encoder was pre-trained through contrastive language video pretraining in the training dataset. The model was trained on a large, single institutional dataset of CMR studies occurring between 2008 and 2022. The dataset was divided into 9,705 training studies, 2,100 validation studies, and 2,094 testing studies. All included studies had left ventricular views (short axis, 4 chamber, 2 chamber long axis, and 3 chamber long axis), and both cine and late gadolinium enhancement (LGE) image types. Only the impressions section was used for this work. We compared our model with several state-of-the-art image captioning models.

Results:

Our approach outperforms multiple previously described methods across all metrics except for CIDEr (Table 1). The BLEU and ROUGE-L metrics, which are statistical correlations of word distributions, show CMR-TARGET generates more of the words from the original report compared to other report generation options. Qualitatively, the generated report achieves more human readable results compared to other models. An example of a test case with CMR-TARGET is displayed in Figure 2.

Conclusion:

Our study demonstrates that the proposed deep learning model generates CMR reports with high accuracy, outperforming baseline models, and thus holds promise for reducing the workload of radiologists and cardiologists and improving clinical efficiency.