Large Language Model for Automated Impression Sentence Generation from Findings (RF_SA_469)

Writing the impression for cardiac magnetic resonance imaging (CMR) studies is a highly intricate task given the need to synthesize information for multiple image types and views, the clinical history of the patient, and knowledge of appropriate interventions. This process is time-intensive and has high demands on expertise. The aim of this study is to develop a method for automating the generation of impression sentences directly from the indication and the findings in CMR reports, thereby streamlining the reporting process and enhancing clinical efficiency.

Methods:

The Large Language Model Meta AI 3 8B (Llama 3) was fine-tuned on a dataset of CMR reports which were parsed into indications, findings, and impressions (Figure 1). This dataset comprised 27,143 studies for training and 5714 studies for testing. We used the LoRA (Low-Rank Adaptation) technique to optimize memory efficiency during training. The training objective was to generate concise impression sentences from the findings in the reports, guided by the clinical indications for each case. A prompt, "Below is an indication for a CMR and the corresponding findings. Write an impression sentence that accurately summarizes the findings based on the indication," was utilized to instruct the model. We evaluated the model based on commonly utilized metrics, BLEU-n, METEOR, ROUGE-L, CIDEr, and BERTScore.

Results:

The results of both the Zero-shot and Fine-tuned approaches are detailed in Table 1. The fine-tuned model demonstrated superior performance across all evaluated metrics, indicating that fine-tuning is essential for optimizing the model's ability to accurately generate impression sentences from CMR findings. An example of a test case with Fine-tuned model is displayed in Figure 2. These results suggest that fine-tuning significantly enhances the model's understanding of the specific nuances required for high-quality clinical report generation.

Conclusion:

We demonstrated Llama 3 model for generating impression sentences from CMR reports. The Fine-tuned model represents a promising tool for streamlining the radiology/cardiology reporting process, potentially improving both efficiency and consistency in clinical practice.