Denoising Diffusion Probabilistic Models
Abstract
We present high quality image synthesis results using diffusion probabilistic models, a class of latent variable models inspired by considerations from nonequilibrium thermodynamics. Our best results are obtained by training on a weighted variational bound designed according to a novel connection between diffusion probabilistic models and denoising score matching with Langevin dynamics, and our models naturally admit a progressive lossy decompression scheme that can be interpreted as a generalization of autoregressive decoding. On the unconditional CIFAR10 dataset, we obtain an Inception score of 9.46 and a state-of-the-art FID score of 3.17. On 256x256 LSUN, we obtain sample quality similar to ProgressiveGAN. Our implementation is available at https://github.com/hojonathanho/diffusion
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
This paper introduces a simplified, noise-prediction training objective for diffusion probabilistic models that bridges variational inference with denoising score matching, enabling high-fidelity, stable image generation that rivals GANs. By rigorously deriving the connection to Langevin dynamics, providing comprehensive ablations, and releasing open-source implementations, the work establishes a new foundational paradigm in generative modeling that has since become the dominant architecture for large-scale synthesis across multiple modalities.
Methodology
The paper presents a rigorous and elegant reformulation of diffusion probabilistic models. The core methodological breakthrough lies in the reparameterization of the reverse process mean to predict the added noise $\epsilon$ rather than the posterior mean or the original data $\mathbf{x}_0$. This choice is theoretically motivated by a novel connection to denoising score matching and annealed Langevin dynamics, which simplifies the variational bound into a straightforward, unweighted mean-squared error objective across timesteps. The authors carefully derive the variance-reduced ELBO, justify the fixed variance schedule, and introduce a discrete decoder for tractable log-likelihood estimation. The mathematical framing is exceptionally clear, transforming a previously cumbersome latent variable model into a highly practical and stable training paradigm.
Results & evaluation
The experimental design is thorough and well-calibrated. The authors evaluate on CIFAR-10, CelebA-HQ, and multiple LSUN categories, achieving state-of-the-art FID (3.17) and Inception Score (9.46) on unconditional CIFAR-10, directly competing with and surpassing contemporary GANs. The ablation studies in Table 2 are particularly strong, isolating the impact of the $\epsilon$-prediction parameterization, the simplified objective, and learned vs. fixed variances. The progressive lossy compression analysis and interpolation experiments provide valuable qualitative and quantitative insights into the model's latent structure and rate-distortion behavior. While log-likelihoods lag behind autoregressive models and flows, the authors correctly attribute this to the model's inductive bias toward perceptual quality over exact density estimation.
Reproducibility
Excellent. The paper provides exhaustive implementation details: U-Net architecture specifications, group normalization, sinusoidal time embeddings, self-attention placement, dropout rates, optimizer settings, learning rate schedules, EMA decay, batch sizes, and exact training steps per dataset. Hardware (TPU v3-8) and training/sampling times are documented. The complete codebase is publicly released, and hyperparameter choices are justified with brief sweep results. This level of transparency makes exact replication highly feasible.
Limitations
The primary limitation is sampling speed: generating a single image requires 1000 sequential neural network evaluations, making it orders of magnitude slower than GANs or single-pass VAEs. The paper acknowledges this but does not propose acceleration techniques (which later work like DDIM would address). Additionally, the lossy compression framework relies on minimal random coding, which is theoretically sound but computationally intractable for high-dimensional data, limiting its immediate practical utility for compression. Log-likelihoods remain uncompetitive with likelihood-maximizing models, and the model's reliance on a fixed, hand-tuned noise schedule could be seen as a hyperparameter bottleneck.
Broader impact
The authors provide a balanced broader impact statement, acknowledging risks of deepfake generation and dataset bias amplification, while highlighting potential benefits in data compression, representation learning, and creative applications. The methodological shift toward diffusion-based generation has since catalyzed massive advancements in text-to-image synthesis, video generation, and molecular design, fundamentally altering the trajectory of AI research and deployment. This paper introduces a simplified, noise-prediction training objective for diffusion probabilistic models that bridges variational inference with denoising score matching, enabling high-fidelity, stable image generation that rivals GANs. By rigorously deriving the connection to Langevin dynamics, providing comprehensive ablations, and releasing open-source implementations, the work establishes a new foundational paradigm in generative modeling that has since become the dominant architecture for large-scale synthesis across multiple modalities.
Curator note
Ho et al.; launched the diffusion model era
Discussion
Talks & videos
Related later work
- Latent Diffusion / Stable Diffusion 2021
- DALL-E 2 2022
- Imagen (Google) 2022
- DiffWave (audio) 2020
- Score-SDE 2020
- RFDiffusion (protein design) 2022
Author of this paper? Send a correction.