Diffusion Reading Group at EleutherAI

This is an ongoing study group occuring the EleutherAI Discord server. You can join the server over here, then head to the "Diffusion Reading Group" thread under the #reading-groups channel.

Here is a playlist of the previous session recordings.

• Diffusion Reading Group at EleutherAI
  • Week 1: DDPM paper
  • Week 2: Score-based generative modeling
  • Week 3: NCSNv2 and Score SDEs
  • Week 4: Score SDEs and DDIM
  • Week 5: IDDPM, ADM, and Classifier-Free Guidance
  • Week 6: Review
  • Week 7: Classifier-Free Guidance, VDMs, and Denoising Diffusion GANs
  • Week 8: Perception-Prioritized Training, Elucidating Design Spaces
  • Week 9: DDPM paper, EDM paper code walk-thrus
  • Week 10: Paella
  • Week 11: Progressive Distillation, Distillation of Guided Models
  • Week 12: SDEDit
  • Week 13: Latent Diffusion and Stable Diffusion
  • Week 14: Q&A with Robin Rombach
  • Week 15: Soft Diffusion
  • Week 16 & 17: Flow Matching
  • Week 18: Consistency Models
  • Week 19: Conditional Flow Matching
  • Week 20: Inverse Heat Dissipation
  • Week 21: Poisson Flow Generative Models
  • Week 22: Min-SNR Weighting Strategy
  • Week 23: PFGM++
  • Week 24: Blurring Diffusion Models
  • Week 25: ControlNet
  • Week 26: DDPO (RLHF Diffusion)
  • Week 27: Diffusion Transformers
  • Week 28: simple diffusion
  • Week 29: Wuerstchen
  • Week 30: Palette
  • List of papers to cover:

Week 1: DDPM paper

• Readings:
  • Denoising Diffusion Probabilistic Models
  • AssemblyAI blog post
• Recording
• Slides

Week 2: Score-based generative modeling

• Readings:
  • Generative Modeling by Estimating Gradients of the Data Distribution
  • Yang Song's blog post
• Recording
• Slides

Week 3: NCSNv2 and Score SDEs

• Readings:
  • Improved Techniques for Training Score-Based Generative Models
  • Score-Based Generative Modeling through Stochastic Differential Equations
  • Yang Song's blog post
• Recording
• Slides

Week 4: Score SDEs and DDIM

• Readings:
  • Score-Based Generative Modeling through Stochastic Differential Equations
  • Yang Song's blog post
  • Denoising Diffusion Implicit Models
  • Lilian Weng's blog post
• Recording
• Slides

Week 5: IDDPM, ADM, and Classifier-Free Guidance

• Readings:
  • Improved Denoising Diffusion Probabilistic Models
  • Diffusion Models Beat GANs on Image Synthesis
  • Classifier-Free Diffusion Guidance
  • Sander Dieleman's blog post on guidance
• Recording
• Slides

Week 6: Review

• Readings:
  • All prior readings
• Recording
• Slides

Week 7: Classifier-Free Guidance, VDMs, and Denoising Diffusion GANs

• Readings:
  • Classifier-Free Diffusion Guidance
  • Sander Dieleman's blog post on guidance
  • Variational Diffusion Models
  • Variational Diffusion Model Colab Notebook
  • Calvin Luo's blog post
  • Tackling the Generative Learning Trilemma with Denoising Diffusion GANs
  • Vahdat and Kreis's blog post
  • Arash Vahdat's DLCT talk slides
• Recording
• Slides

Week 8: Perception-Prioritized Training, Elucidating Design Spaces

• Readings:
  • Perception Prioritized Training of Diffusion Models
    • Official Codebase
  • Elucidating the Design Space of Diffusion-Based Generative Models
    • Official Codebase
    • Katherine's codebase
  • Recording
  • Notebook - P2 paper (presented by Molly Beavers)
  • Slides - EDM paper

Week 9: DDPM paper, EDM paper code walk-thrus

• Readings:
  • Denoising Diffusion Probabilistic Models
  • Elucidating the Design Space of Diffusion-Based Generative Models
    • Katherine's codebase (walk-thru by Jeremy Howard)
• Recording
• DDPM Colab notebook

Week 10: Paella

• Readings:
  • Fast Text-Conditional Discrete Denoising on Vector-Quantized Latent Spaces
• Recording

Week 11: Progressive Distillation, Distillation of Guided Models

• Readings:
  • Progressive Distillation for Fast Sampling of Diffusion Models
  • On Distillation of Guided Diffusion Models
• Recording
• Slides - Progressive Distillation - Presented by marii
• Slides - Distillation of Guided Models - Presented by Griffin Floto

Week 12: SDEDit

• Readings:
  • SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
• Recording
• Slides

Week 13: Latent Diffusion and Stable Diffusion

• Readings:
  • High-Resolution Image Synthesis with Latent Diffusion Models
  • Annotated Paper Implementation of Latent/Stable Diffusion
  • Lilian Weng's blog post
  • Stable Diffusion
  • HuggingFace blog post on Stable Diffusion
• Recording
• Slides

Week 14: Q&A with Robin Rombach

• Readings:
  • Same as Week 13
• Recording

Week 15: Soft Diffusion

• Readings:
  • Soft Diffusion: Score Matching for General Corruptions
• Recording

Week 16 & 17: Flow Matching

• Readings:
  • Flow Matching for Generative Modeling
• Recording
• Recording
• Slides

Week 18: Consistency Models

• Readings:
  • Consistency Models
• Recording
• Slides

Week 19: Conditional Flow Matching

• Readings:
  • Conditional Flow Matching: Simulation-Free Dynamic Optimal Transport
• Recording

Week 20: Inverse Heat Dissipation

• Readings:
  • Generative Modelling With Inverse Heat Dissipation
• Recording
• Slides

Week 21: Poisson Flow Generative Models

• Readings:
  • Poisson Flow Generative Models
  • AssemblyAI blog post
• Recording
• Slides
• Slides - other formats

Week 22: Min-SNR Weighting Strategy

• Readings:
  • Efficient Diffusion Training via Min-SNR Weighting Strategy
• Recording

Week 23: PFGM++

• Readings:
  • PFGM++: Unlocking the Potential of Physics-Inspired Generative Models
• Recording
• Slides
• Slides - other formats

Week 24: Blurring Diffusion Models

• Readings:
  • Blurring Diffusion Models
• Recording

Week 25: ControlNet

• Readings:
  • Adding Conditional Control to Text-to-Image Diffusion Models
• Recording

Week 26: DDPO (RLHF Diffusion)

• Readings:
  • Training Diffusion Models with Reinforcement Learning
• Recording
• Slides

Week 27: Diffusion Transformers

• Readings:
  • Scalable Diffusion Models with Transformers
• Recording

Week 28: simple diffusion

• Readings:
  • simple diffusion: End-to-end diffusion for high resolution images
• Recording

Week 29: Wuerstchen

• Readings:
  • Wuerstchen: Efficient Pretraining of Text-to-Image Models
• Recording

Week 30: Palette

• Readings:
  • Palette: Image-to-Image Diffusion Models
• Recording

List of papers to cover:

1. Denoising Diffusion Probabilistic Models
2. Generative Modeling by Estimating Gradients of the Data Distribution
3. Improved techniques for training score-based generative models
4. Score-Based Generative Modeling through Stochastic Differential Equations
5. Denoising Diffusion Implicit Models
6. Improved Denoising Diffusion Probabilistic Models
7. Diffusion Models Beat GANs on Image Synthesis
8. Classifier-Free Diffusion Guidance
9. Variational Diffusion Models
10. Tackling the Generative Learning Trilemma with Denoising Diffusion GANs
11. Perception Prioritized Training of Diffusion Models
12. Elucidating the Design Space of Diffusion-Based Generative Models
13. Fast Text-Conditional Discrete Denoising on Vector-Quantized Latent Spaces
14. Progressive Distillation for Fast Sampling of Diffusion Models
15. On Distillation of Guided Diffusion Models
16. SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
17. High-Resolution Image Synthesis with Latent Diffusion Models
18. Stable Diffusion
19. Soft Diffusion: Score Matching for General Corruptions
20. Flow Matching for Generative Modeling
21. Consistency Models
22. Conditional Flow Matching: Simulation-Free Dynamic Optimal Transport
23. Generative Modelling With Inverse Heat Dissipation
24. Poisson Flow Generative Models
25. Efficient Diffusion Training via Min-SNR Weighting Strategy
26. PFGM++: Unlocking the Potential of Physics-Inspired Generative Models
27. Blurring Diffusion Models
28. Adding Conditional Control to Text-to-Image Diffusion Models (ControlNet)
29. Training Diffusion Models with Reinforcement Learning
30. Scalable Diffusion Models with Transformers
31. simple diffusion: End-to-end diffusion for high resolution images
32. Wuerstchen: Efficient Pretraining of Text-to-Image Models
33. Palette: Image-to-image diffusion models
34. Reflected Diffusion Models
35. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
36. Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise
37. Pyramidal Denoising Diffusion Probabilistic Models
38. Cascaded Diffusion Models for High Fidelity Image Generation
39. Image super-resolution via iterative refinement
40. Denoising Diffusion Probabilistic Models for Robust Image Super-Resolution in the Wild
41. I$^2$SB: Image-to-Image Schr"odinger Bridge
42. GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
43. Hierarchical Text-Conditional Image Generation with CLIP Latents
44. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
45. eDiffi: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers
46. Semi-Parametric Neural Image Synthesis
47. On the Importance of Noise Scheduling for Diffusion Models
48. Pseudo Numerical Methods for Diffusion Models on Manifolds
49. DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Step
50. DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models
51. GENIE: Higher-Order Denoising Diffusion Solvers
52. Journey to the BAOAB-limit: finding effective MCMC samplers for score-based models
53. Riemannian Score-Based Generative Modeling
54. DiffWave: A Versatile Diffusion Model for Audio Synthesis
55. Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech
56. Video diffusion models
57. MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation
58. Imagen Video: High Definition Video Generation with Diffusion Models
59. Make-A-Video: Text-to-Video Generation without Text-Video Data
60. DreamFusion: Text-to-3D using 2D Diffusion
61. Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation
62. Magic3D: High-Resolution Text-to-3D Content Creation
63. Diffusion-LM Improves Controllable Text Generation.
64. Autoregressive Diffusion Models
65. Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning
66. Continuous diffusion for categorical data
67. DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
68. Vector quantized diffusion model for text-to-image synthesis
69. Improved Vector Quantized Diffusion Models
70. DiVAE: Photorealistic Images Synthesis with Denoising Diffusion Decoder
71. Diffusion Models already have a Semantic Latent Space
72. Understanding ddpm latent codes through optimal transport
73. Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling
74. Dual Diffusion Implicit Bridges for Image-to-Image Translation
75. Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance
76. Zero-shot Image-to-Image Translation