Multi-Stage Patch Diffusion and Contrastive-SDE

<p dir="ltr">In this work, we address two key problems in diffusion models: (1) Efficiency of diffusion models, and (2) unpaired image-to-image (I2I) translation. Score-based diffusion models have demonstrated state-of-the-art performance in generative tasks. Their ability to approximate complex data distributions through stochastic differential equations (SDEs) enables them to generate high-fidelity and diverse outputs, making them particularly effective for unconditional image generation and unpaired I2I translation. To enhance efficiency, we propose an integrated approach that combines Patch Diffusion with a Multi-Stage Framework to enhance both training efficiency and generative quality. Patch Diffusion introduces a patch-wise training methodology that employs randomized patch sizes and conditional score functions with location-based coordinate channels, enabling faster convergence and improved performance on smaller datasets. The Multi-Stage Framework, on the other hand, optimizes parameter utilization across different diffusion timesteps by segmenting them into distinct intervals, employing tailored multi-decoder U-Net architectures, and addressing inefficiencies such as gradient dissimilarity. For unpaired I2I translation, we propose a time-based contrastive learning method, Contrastive SDE, where a model is trained using SimCLR by treating an image and its domain-invariant representation as a positive pair. This helps the model retain features consistent across domains while suppressing those specific to individual domains. The trained contrastive model is then used to guide the inference process of a pretrained SDE for image-to-image translation. Experimental results demonstrate a substantial reduction in training time, although the FID scores remain suboptimal across datasets such as CIFAR10 and CelebA, highlighting both the promise and current limitations of the proposed framework in democratizing diffusion model training for broader applications. We further conduct empirical comparisons of Contrastive SDE with several baselines across three standard unpaired I2I tasks, evaluated using four common metrics. Contrastive SDE achieves performance comparable to state-of-the-art methods on several metrics. Moreover, our model converges significantly faster and requires neither label supervision nor classifier training, making it a more efficient alternative for unpaired image-to-image translation.</p>