Skip to content
Last Dictionary
Plane
Generative MediaInference & EfficiencyTraining & Post-training

diffusion distillation

nounAlso: step distillation, few-step diffusion, progressive distillation, adversarial diffusion distillation, distribution matching distillation, turbo model

In plain wordsTraining a fast image generator to imitate a slow diffusion model, so that a picture that once took fifty steps can be made in one to four.

  1. A family of methods, a form of knowledge distillationknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run., that compress a trained diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static. or flow matchingflow matching, n.: A way to train generators that learns, for every point between random noise and real data, which direction to move, so that noise can be steered smoothly into new examples. model needing tens of sampling steps into a student that generates in one to a few network evaluations. Progressive distillation repeatedly trains a student to match two teacher steps with one, halving the step count each round. Consistency distillationconsistency model, n.: A generator trained to jump from noise straight to a finished image in one step, instead of the dozens of small steps a diffusion model normally takes. trains a model to map any point on a teacher trajectory to its endpoint. Distribution-matching methods train a one-step generator so that its output distribution matches the teacher's, using the difference between two score estimates as a gradient (distribution matching distillation) or combining score distillation with a GAN-style discriminator (adversarial diffusion distillation, behind 'Turbo' models). Guidanceclassifier-free guidance, n.: A setting that makes an image generator follow the prompt more strictly by comparing what it would draw with and without the prompt, then exaggerating the difference. can be distilled at the same time, removing its doubled cost. Distilled models make real-time and on-device generation practical but can lose diversity, fine control and negative promptnegative prompt, n.: A second prompt listing what you do not want in a generated image — 'blurry, extra fingers, watermark' — which the generator then steers away from. support.

Origin

Tim Salimans and Jonathan Ho's progressive distillation (posted February 2022; ICLR 2022) reached four-step sampling. Consistency models (2023), adversarial diffusion distillation by Axel Sauer and colleagues at Stability AI (November 2023) and distribution matching distillation by Tianwei Yin and colleagues (November 2023) pushed toward a single step.

Related

See also
  • consistency modelconsistency model, n.: A generator trained to jump from noise straight to a finished image in one step, instead of the dozens of small steps a diffusion model normally takes.
  • knowledge distillationknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run.
  • diffusion samplerdiffusion sampler, n.: The step-by-step routine a diffusion model follows to turn noise into a finished picture; the choice of sampler and number of steps trades speed against quality.
  • score distillation samplingscore distillation sampling, n.: A trick for creating 3D objects from a text description by repeatedly asking a 2D image generator whether pictures of the object, seen from many angles, look right — and adjusting the object until they do.
  • generative adversarial networkgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people.
  • rectified flowrectified flow, n.: A generator trained to move random noise toward real data along straight lines, which can then be straightened further so that a picture comes out in one or a few steps.

Sources

  1. Salimans, T. & Ho, J. (2022). Progressive Distillation for Fast Sampling of Diffusion Models. ICLR 2022. arXiv:2202.00512. arxiv.org/abs/2202.00512
  2. Sauer, A., Lorenz, D., Blattmann, A. & Rombach, R. (2023). Adversarial Diffusion Distillation. arXiv:2311.17042. arxiv.org/abs/2311.17042
  3. Yin, T. et al. (2024). One-step Diffusion with Distribution Matching Distillation. CVPR 2024. arXiv:2311.18828. arxiv.org/abs/2311.18828

Mentioned in

diffusion samplerdiffusion sampler, n.: The step-by-step routine a diffusion model follows to turn noise into a finished picture; the choice of sampler and number of steps trades speed against quality., FLUXFLUX, prop. n.: A family of image-generating AI models from the German start-up Black Forest Labs, founded by researchers behind Stable Diffusion; some versions can be downloaded and run freely., generative adversarial networkgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people., consistency modelconsistency model, n.: A generator trained to jump from noise straight to a finished image in one step, instead of the dozens of small steps a diffusion model normally takes., rectified flowrectified flow, n.: A generator trained to move random noise toward real data along straight lines, which can then be straightened further so that a picture comes out in one or a few steps., score distillation samplingscore distillation sampling, n.: A trick for creating 3D objects from a text description by repeatedly asking a 2D image generator whether pictures of the object, seen from many angles, look right — and adjusting the object until they do.