Skip to content
Last Dictionary
Plane
Generative MediaInference & Efficiency

consistency model

nounAlso: consistency distillation, consistency training, latent consistency model, LCM, LCM-LoRA

In plain wordsA generator trained to jump from noise straight to a finished image in one step, instead of the dozens of small steps a diffusion model normally takes.

  1. A generative modelgenerative model, n.: A model that learns what a kind of data looks like — faces, sentences, songs — well enough to produce new examples that resemble the originals without copying them. that learns a function mapping any point on a trajectory of a diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static.'s probability-flow ODE directly to the trajectory's clean endpoint. The defining self-consistency property — all points on the same trajectory map to the same output — is enforced by training the network so that its outputs at adjacent noise levels agree, with a boundary condition that makes it the identity at zero noise. A consistency model can generate with a single network evaluation, or spend a few more evaluations for better samples by alternately re-noising and re-mapping. It can be trained by consistency distillation, using a pretrained diffusion model's ODE solver to produce adjacent points, or by consistency training directly from data. Latent consistency models (2023) applied the idea to Stable Diffusion, producing images in two to four steps, and continuous-time variants (2024) scaled it further.

Origin

Introduced by Yang Song, Prafulla Dhariwal, Mark Chen and Ilya Sutskever of OpenAI in a paper posted in March 2023 (ICML 2023), which reported one-step FIDs of 3.55 on CIFAR-10 and 6.20 on ImageNet 64 × 64. Simian Luo and colleagues introduced latent consistency models in October 2023.

Related

See also
  • diffusion distillationdiffusion distillation, n.: Training a fast image generator to imitate a slow diffusion model, so that a picture that once took fifty steps can be made in one to four.
  • diffusion samplerdiffusion sampler, n.: The step-by-step routine a diffusion model follows to turn noise into a finished picture; the choice of sampler and number of steps trades speed against quality.
  • diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static.
  • rectified flowrectified flow, n.: A generator trained to move random noise toward real data along straight lines, which can then be straightened further so that a picture comes out in one or a few steps.
  • score-based generative modelscore-based generative model, n.: A generator that learns, for any noisy example, which direction would make it look more like real data — then starts from pure noise and follows those directions step by step.
  • knowledge distillationknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run.

Sources

  1. Song, Y., Dhariwal, P., Chen, M. & Sutskever, I. (2023). Consistency Models. ICML 2023. arXiv:2303.01469. arxiv.org/abs/2303.01469
  2. Luo, S., Tan, Y., Huang, L., Li, J. & Zhao, H. (2023). Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference. arXiv:2310.04378. arxiv.org/abs/2310.04378

Mentioned in

diffusion samplerdiffusion sampler, n.: The step-by-step routine a diffusion model follows to turn noise into a finished picture; the choice of sampler and number of steps trades speed against quality., denoising diffusion implicit modeldenoising diffusion implicit model, n.: A faster way to run a diffusion image generator that skips most of the steps and, for the same starting noise, always gives the same picture., diffusion distillationdiffusion distillation, n.: Training a fast image generator to imitate a slow diffusion model, so that a picture that once took fifty steps can be made in one to four., diffusion policydiffusion policy, n.: A way of training robots in which the same kind of AI that generates images instead generates the robot's next movements, letting it copy demonstrations even when there are several good ways to do a task., rectified flowrectified flow, n.: A generator trained to move random noise toward real data along straight lines, which can then be straightened further so that a picture comes out in one or a few steps.