High-Frame-Rate Exercise CMR Real-Time Flow using Transformer-Based Deep Learning Interpolator

Exercise cardiovascular magnetic resonance (Ex-CMR) real-time phase-contrast imaging enables flow quantification after physiological exercise. However, a tradeoff exists between spatial and temporal resolution. Insufficient temporal resolution limits Ex-CMR flow quantification during exercise. Therefore, we aimed to develop a deep learning model to increase real-time temporal resolution for Ex-CMR while maintaining spatial resolution.

Methods:

We developed a deformation encoding transformer (DENT) model for high-frame-rate flow imaging (Fig. 1) and applied it to a real-time flow sequence to achieve a 2-fold increase in temporal resolution and a 4-fold increase in framerate. The inputs are T = 4 velocity-compensated v₀ or encoded v_± images with temporal resolution Δt, acquired at times t - Δt, t, t + Δt and t + 2Δt. The output is an image interpolated in-between at t + 0.5Δt. DENT has six pairs of transformer layers whose inputs are split into windows along spatial and temporal dimensions. Spatiotemporal associations within windows were learned by applying self-attention to each window. These associations were used to predict non-rigid motion components, which were used to register the T frames to a single time point t + 0.5Δt. Breath-hold, 2D electrocardiogram (ECG)-gated flow images collected in multiple views from 1600 patients referred for clinical CMR were used for training. Velocity-compensated and encoded images with downsampled framerates were restored using DENT and compared to ground truth using L₁ loss.  In a prospective study of 12 patients referred for clinical CMR, a set of breath-hold, 2D ECG-gated flow for comparison and set of free-breathing, 2D real-time velocity-encoded (two-sided: v_-, v₊) images were collected in the aortic view with resolution of 2.5 × 1.9 mm² and 44 ms. In a prospective Ex-CMR study of 8 healthy subjects, real-time images were collected before and after exercise. Phase-contrast data were obtained from two-sided v_± images using conventional and view-sharing methods [1,2], resulting in flow images with an actual 88 ms resolution and framerates of 11 fps and 23 fps (Fig. 1B). DENT 4-fold reconstruction provided a true 44 ms resolution with 46 fps framerate.

Results:

DENT removed flickering artifacts caused by view-sharing and improved quantification of flow (Fig. 2). Biases of peak parameters at rest (Fig. 3A) between ECG-gated and real-time flow (conventional, view-sharing, DENT): flow rate of -35 ± 30 ml/s, -22 ± 29 ml/s, -9 ± 3 ml/s; mean velocity of -8 ± 7 cm/s, -4 ± 5 cm/s, 2 ± 6 cm/s. (Fig.3B) View-sharing underestimated the change in peak flow rate post-exercise (42% vs 35%) relative to DENT, and in one case, inaccurately overestimated the change (Fig.3C). Representative subjects illustrating the benefit of DENT-based higher temporal resolution are shown in Fig.3D.

Conclusion:

DENT enabled a 4-fold gain in framerate and 2-fold gain in temporal resolution relative to conventional reconstruction, improving Ex-CMR flow measures.

Fig. 1 (A) DENT uses four velocity-encoded image frames with Δt resolution to interpolate a frame in between. Encoding transformer layers extract spatiotemporal features. Deconvolutional layers predict non-rigid motion parameters, which are used to sample and combine the four frames to produce the new velocity frame. (B) Real-time phase-contrast data obtained with conventional reconstruction have an effective resolution of 2∆t and framerate 1/2∆t. View-sharing doubles the framerate to 1/∆t. DENT doubles both framerate and temporal resolution. Repeating this process (DENT 4-fold) before phase-contrast reconstruction would provide a 4-fold gain in framerate.

Fig. 2. (A) Imaging parameters for ECG-gated and real-time two-sided phase-contrast imaging. Real-time images had a compressed sensing rate of 9, yielding a temporal resolution of 44 ms. (B) Conventional reconstruction of phase-contrast images from velocity-encoded data resulted in a true temporal resolution of 88 ms. View-sharing doubled the framerate, while DENT improved both framerate and resolution. Flickering artifacts in view-sharing appeared as intensity variations in temporal profile (white arrow). (C) Aortic (AO) flow rate was similar at rest for both methods and increased post-exercise. View-sharing underestimated the post-exercise peak (red arrow).

Figure 3. (A) Comparison of resting peak flow rate between ECG-gated and real-time flow with conventional, view-sharing, and DENT methods (Fig. 1B). Conventional reconstruction produces one phase-contrast image for every velocity-encoded negative and positive pair (Fig. 1B). Red and blue lines indicate at least a 5% difference between view-sharing and DENT. (B) Changes in peak flow parameters after exercise. As shown in C, the single blue line reflects view-sharing's underestimation at rest (blue arrows) but not during exercise (blue and red arrows), which results in an exaggerated pre- and post-exercise change that is not real. (D) Representative subjects highlight the benefit of using DENT instead of view-sharing reconstruction.