[Review] Diffusion Models 3 - 작성 중

Song et al., 2020의 Score-Based Generative Modeling through Stochastic Differential Equations 논문에서 우리는 Diffusion forward process가 forward SDE로 나타나고, reverse process는 대응하는 reverse SDE를 푸는 것을 알았다. 또한, reverse SDE에 대응되는 Probability Flow ODE를 통해 빠른 Sampling이 가능했다. black-box ODE Solver로 RK45를 사용했는데, 이게 최선일까? Sampling은 더 빠르면서도 fidelity는 여전히 높은 ODE Solver는 어떤 것이 있을까? 아래의 논문들을 살펴보자.

1. Zhang & Chen (2022). Fast sampling of diffusion models with exponential integrator.
2. Lu et al. (2022). Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps.
3. Dockhorn et al. (2022). Genie: Higher-order denoising diffusion solvers.
4. Karras et al. (2022). Elucidating the design space of diffusion-based generative models.
5. Liu et al. (2022). Pseudo Numerical Methods for Diffusion Models on Manifolds
6. Bao et al. (2022). Analytic-dpm: an analytic estimate of the optimal reverse variance in diffusion probabilistic models.
7. Kong & Ping (2021). On fast sampling of diffusion probabilistic models.
8. Tachibana et al. (2022). Quasi-Taylor Samplers for Diffusion Generative Models based on Ideal Derivatives

0. Recap

Summary

Reference

Websites

[사이트 출처 1]

Papers

[1] Zhang, Q., & Chen, Y. (2022). Fast sampling of diffusion models with exponential integrator. arXiv preprint arXiv:2204.13902.

[2] Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., & Zhu, J. (2022). Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35, 5775-5787.

[3] Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., & Zhu, J. (2022). Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095.

[4] Dockhorn, T., Vahdat, A., & Kreis, K. (2022). Genie: Higher-order denoising diffusion solvers. Advances in Neural Information Processing Systems, 35, 30150-30166.

[5] Karras, T., Aittala, M., Aila, T., & Laine, S. (2022). Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems, 35, 26565-26577

[6] Liu, L., Ren, Y., Lin, Z., & Zhao, Z. (2022). Pseudo numerical methods for diffusion models on manifolds. arXiv preprint arXiv:2202.09778.

[7] Bao, F., Li, C., Zhu, J., & Zhang, B. (2022). Analytic-dpm: an analytic estimate of the optimal reverse variance in diffusion probabilistic models. arXiv preprint arXiv:2201.06503.

[8] Kong, Z., & Ping, W. (2021). On fast sampling of diffusion probabilistic models. arXiv preprint arXiv:2106.00132.