Spectral Bias in Training of Diffusion Model

Additional figures and results for Spectral Bias in Training of Diffusion Model.

We included empirical results regarding the learning dynamics of diffusion model. There are three kinds of experimetal results, noted by their file name

• Figure_MLP_UNet_validation_[dataset]. These are MLP-based UNet trained on various natural image datasets, by flatten the images as vectors. We tracked the generated image variance along eigenvectors of training dataset. We noted a robust spectral bias as predicted by the analytical theory, and the power coefficient is indeed close to -1 for the top eigenmodes.
• Figure_CNN_UNet_full_img_validation_[dataset]. These are CNN-based UNet trained on various natural image datasets, comparable to those used in MLP experiments. We tracked the generated image variance along eigenvectors of training dataset. We noted the spectral bias is not evident in the learning dynamics regarding full image.
• Figure_supp_CNN_validation_[dataset]. These are CNN-based UNet trained on various natural image datasets. We tracked the variance of patches in generated samples along eigenvectors of the patches extracted from training set. We noted the spectral bias is more obvious regarding the patches.

Figure_MLP_UNet_validation_[dataset]. A, B, shows the generated variance along eigenmodes of the whole image, A shows eigenmodes 5-95, while B shows eigenmodes 0-900. Left panel showing unnormalized variance, right panel showing the variance normalized by the target value; C. shows the scaling relationship between emergence time vs target variance with fitted line on log log scale. Below, we plot sample images throughout training to give a visual sense of learning progress.

Figure_CNN_UNet_full_img_validation_[dataset]. The exact same plotting format as above, but for CNN-based UNet trained on the same datasets.

Figure_supp_CNN_validation_[dataset]. Each row shows a different patch size, stride size. In each row, left panel shows the dynamics of generated variance along eigenmodes for patches, right panel shows the emergence time vs target variance for these patches.