A patch-based latent video diffusion model for cardiac cine (Cine-LDiff) (RF_TH_167)

Presenting Author(s)

• XL

  Xuan Lei, BSc

  Graduate Research Assistant
  The Ohio State University

Primary Author(s)

• XL

  Xuan Lei, BSc

  Graduate Research Assistant
  The Ohio State University

Co-Author(s)

• PS

  Philip Schniter, PhD

  Professor
  The Ohio State University
• 

  Rizwan Ahmad, PhD

  Associate Professor
  The Ohio State University

Figure 1: Training and sample generation using Cine-LDiff. (A) Spatiotemporal image patches are extracted for each cine series. (B) The cine patches are compressed to latent patches using a variational autoencoder. (C) A video diffusion model (based on a denoising U-Net that accepts the diffusion time-step t) is trained on noisy latent patches. (D) Starting from a random Gaussian latent at diffusion time-step T, a new latent sample is generated using T steps of reverse DDPM diffusion and finally converted to the voxel domain.

Figure 2: Examples of one temporal frame from real cine patches (top) and generated cine patches (bottom). The generated samples were chosen to roughly match the anatomical locations of the real samples.

Figure 3: Five 128x8 spatial-temporal profiles from the real (left) and generated (right) patches shown in Figure 2. The profiles were read along a vertical line passing through the heart.


Background: Diffusion models are powerful generative AI models capable of generating high-quality samples, e.g., images or videos, from distributions observed only through training datasets. Once trained, such models can be used to solve a variety of inverse problems in imaging, including reconstruction, deblurring, and super-resolution. Training diffusion models, however, requires substantial computational resources and large datasets, which are typically not available for cardiovascular MRI (CMR). In this work, we propose a patch-based latent video diffusion model, called Cine-LDiff, for cardiac cine. By using patches, we simultaneously increase the number of training samples and decrease the model complexity. Also, by diffusing in the latent space, we allow a further reduction in computational resources. Our preliminary results show that Cine-LDiff generates spatiotemporal CMR cine patches of very high quality.

Methods: We started with 207 coil-combined complex-valued cine series from the OCMR repository [1], selecting data exclusively from high-field scanners (≥1.5T) to avoid low signal-to-noise ratio series. First, the cine series were converted to 29,068 overlapping patches of size 128x128x8x2, where 128x128 represents the spatial size, 8 represents the number of frames, and 2 represents the real/imaginary channels in a patch. Second, the patches were used to train a variational auto-encoder (AE) [2] that used a factor of four compression for the latent patches. Third, a video diffusion model [3] was trained on the latent patches. Fourth, new latent patches were generated using a reverse diffusion process based on denoising diffusion probabilistic models (DDPM) [4]. Finally, the generated latent patches were passed through the AE decoder to obtain cine patches of size 128x128x8x2. To assess the quality of generated patches, Fréchet inception distance (FID) [5] was computed between the generated dataset and a real dataset, as well as between two real datasets as a baseline, each with 2,320 cine patches. The training and the sample-generation process are summarized in Figure 1.

Results: Figure 2 shows representative frames from five different real and generated cine patches. To highlight the diversity in the generated samples, the samples were hand-picked to approximately match the anatomical locations of the real samples. Figure 3 shows spatial-temporal profiles from the images shown in Figure 2, where cardiac motion is evident. The FID (recall lower is better) between the real and generated datasets was 49.10, and between the two real datasets was 49.66, suggesting that the generated samples are essentially indistinguishable from the real samples in the sense of FID.

Conclusion: We proposed Cine-LDiff, a patch-based latent video diffusion model for cardiac cine. Our ongoing efforts are focused on leveraging Cine-LDiff to facilitate highly accelerated real-time cine imaging.

Figure 1: Training and sample generation using Cine-LDiff. (A) Spatiotemporal image patches are extracted for each cine series. (B) The cine patches are compressed to latent patches using a variational autoencoder. (C) A video diffusion model (based on a denoising U-Net that accepts the diffusion time-step t) is trained on noisy latent patches. (D) Starting from a random Gaussian latent at diffusion time-step T, a new latent sample is generated using T steps of reverse DDPM diffusion and finally converted to the voxel domain.

Figure 2: Examples of one temporal frame from real cine patches (top) and generated cine patches (bottom). The generated samples were chosen to roughly match the anatomical locations of the real samples.

Figure 3: Five 128x8 spatial-temporal profiles from the real (left) and generated (right) patches shown in Figure 2. The profiles were read along a vertical line passing through the heart.