diffusion sampler
In plain wordsThe step-by-step routine a diffusion model follows to turn noise into a finished picture; the choice of sampler and number of steps trades speed against quality.
An algorithm that generates a sample from a trained diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static. by numerically integrating its reverse process from noise to data, calling the network once or more per step. Stochastic samplers simulate the reverse-time stochastic differential equation and inject fresh noise at each step (ancestral sampling, as in the original DDPMdenoising diffusion probabilistic model, n.: The 2020 recipe that made diffusion image generators work well: add noise to pictures over a thousand small steps, then train a network to guess the noise so it can be removed.); deterministic samplers solve the equivalent probability-flow ordinary differential equation, so a given starting noise always yields the same image, which enables inversion and interpolation. Widely used methods include DDIMdenoising diffusion implicit model, n.: A faster way to run a diffusion image generator that skips most of the steps and, for the same starting noise, always gives the same picture., the Euler and Heun steppers of the EDM framework, and higher-order solvers such as DPM-Solver and DPM-Solver++, which reach good quality in roughly 10–25 network evaluations instead of hundreds or thousands. Image tools expose the sampler, step count and random seedrandom seed, n.: A number that fixes the 'random' choices a computer makes; reuse the same seed and settings, and an image generator produces the same picture again. as user settings. Cost is counted in network function evaluations (NFE); diffusion distillationdiffusion distillation, n.: Training a fast image generator to imitate a slow diffusion model, so that a picture that once took fifty steps can be made in one to four. and consistency modelsconsistency model, n.: A generator trained to jump from noise straight to a finished image in one step, instead of the dozens of small steps a diffusion model normally takes. cut it to between one and four.
Twenty steps of DPM-Solver++ looked as good as 250 ancestral steps.
Origin
Diffusion sampling began as ancestral sampling of a Markov chain (2015, 2020). Jiaming Song, Chenlin Meng and Stefano Ermon's DDIM (October 2020) introduced fast deterministic sampling; the SDE framework of Yang Song and colleagues (2020) recast samplers as SDE and ODE solvers; and Tero Karras and colleagues' EDM (2022) and Cheng Lu and colleagues' DPM-Solver (2022) systematized the design space.
Usage note
Sampler names in community tools ('Euler a', 'DPM++ 2M Karras') bundle the solver, whether noise is re-injected ('a' for ancestral) and the spacing of noise levels; they are conventions of particular software, not standard terminology.
Related
- diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static.
- noise schedulenoise schedule, n.: The plan for how much static is mixed into training examples at each step of a diffusion model, from almost none to total noise; its shape affects quality and speed.
- denoising diffusion implicit modeldenoising diffusion implicit model, n.: A faster way to run a diffusion image generator that skips most of the steps and, for the same starting noise, always gives the same picture.
- score-based generative modelscore-based generative model, n.: A generator that learns, for any noisy example, which direction would make it look more like real data — then starts from pure noise and follows those directions step by step.
- consistency modelconsistency model, n.: A generator trained to jump from noise straight to a finished image in one step, instead of the dozens of small steps a diffusion model normally takes.
- diffusion distillationdiffusion distillation, n.: Training a fast image generator to imitate a slow diffusion model, so that a picture that once took fifty steps can be made in one to four.
- random seedrandom seed, n.: A number that fixes the 'random' choices a computer makes; reuse the same seed and settings, and an image generator produces the same picture again.
Sources
- Karras, T., Aittala, M., Aila, T. & Laine, S. (2022). Elucidating the Design Space of Diffusion-Based Generative Models. NeurIPS 2022. arXiv:2206.00364. arxiv.org/abs/2206.00364
- Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C. & Zhu, J. (2022). DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps. NeurIPS 2022. arXiv:2206.00927. arxiv.org/abs/2206.00927
Mentioned in
diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static., classifier-free guidanceclassifier-free guidance, n.: A setting that makes an image generator follow the prompt more strictly by comparing what it would draw with and without the prompt, then exaggerating the difference., random seedrandom seed, n.: A number that fixes the 'random' choices a computer makes; reuse the same seed and settings, and an image generator produces the same picture again., samplingsampling, n.: Picking some items at random from a larger set or from a range of possibilities — surveying 1,000 voters, say, or letting an AI choose its next word by a weighted lottery., consistency modelconsistency model, n.: A generator trained to jump from noise straight to a finished image in one step, instead of the dozens of small steps a diffusion model normally takes., denoising diffusion implicit modeldenoising diffusion implicit model, n.: A faster way to run a diffusion image generator that skips most of the steps and, for the same starting noise, always gives the same picture., diffusion distillationdiffusion distillation, n.: Training a fast image generator to imitate a slow diffusion model, so that a picture that once took fifty steps can be made in one to four., noise schedulenoise schedule, n.: The plan for how much static is mixed into training examples at each step of a diffusion model, from almost none to total noise; its shape affects quality and speed., score-based generative modelscore-based generative model, n.: A generator that learns, for any noisy example, which direction would make it look more like real data — then starts from pure noise and follows those directions step by step.