Skip to content
Last Dictionary
Plane
Generative MediaDeep Learning

noise schedule

nounAlso: variance schedule, diffusion schedule, beta schedule, cosine schedule, zero terminal SNR, v-prediction

In plain wordsThe plan for how much static is mixed into training examples at each step of a diffusion model, from almost none to total noise; its shape affects quality and speed.

  1. The function that fixes how much signal and how much Gaussian noise a diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static.'s noisy input contains at each timestep or noise level, usually expressed through β_t, through the cumulative signal fraction ᾱ_t, or as a signal-to-noise ratio (SNR) that falls from very high to near zero. The original DDPMdenoising diffusion probabilistic model, n.: The 2020 recipe that made diffusion image generators work well: add noise to pictures over a thousand small steps, then train a network to guess the noise so it can be removed. used a linear β schedule over 1,000 steps; the cosine schedule (2021) destroys information more gradually and improved results at low resolution, while higher resolutions need schedules shifted toward more noise, because redundancy between neighboring pixels makes images easier to recover. Training and sampling choices interact: schedules whose final step still leaks signal create a train–test mismatch, which in Stable Diffusion limited very bright or dark images and was fixed by enforcing zero terminal SNR. Related design choices include the prediction target (noise, clean data or 'v', a mixture of the two) and how timesteps are weighted or sampled in training.

Origin

Explicit variance schedules appeared with diffusion probabilistic models (Sohl-Dickstein and colleagues, 2015) and DDPM (2020). Alex Nichol and Prafulla Dhariwal introduced the cosine schedule in 2021; Tero Karras and colleagues' EDM paper (2022) recast schedules as choices of noise levels σ; and Shanchuan Lin and colleagues (2023) showed that common schedules fail to reach zero terminal SNR.

Usage note

Not to be confused with a learning rate schedulelearning rate schedule, n.: A plan for changing how big a model's learning steps are during training — usually starting cautiously, then taking big steps, then smaller and smaller ones as it settles in.: a noise schedule shapes how training data are corrupted, not the optimizer's step size.

Related

See also
  • diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static.
  • denoising diffusion probabilistic modeldenoising diffusion probabilistic model, n.: The 2020 recipe that made diffusion image generators work well: add noise to pictures over a thousand small steps, then train a network to guess the noise so it can be removed.
  • diffusion samplerdiffusion sampler, n.: The step-by-step routine a diffusion model follows to turn noise into a finished picture; the choice of sampler and number of steps trades speed against quality.
  • rectified flowrectified flow, n.: A generator trained to move random noise toward real data along straight lines, which can then be straightened further so that a picture comes out in one or a few steps.
  • learning rate schedulelearning rate schedule, n.: A plan for changing how big a model's learning steps are during training — usually starting cautiously, then taking big steps, then smaller and smaller ones as it settles in.
Contrast with
  • learning rate schedulelearning rate schedule, n.: A plan for changing how big a model's learning steps are during training — usually starting cautiously, then taking big steps, then smaller and smaller ones as it settles in.

Sources

  1. Nichol, A. & Dhariwal, P. (2021). Improved Denoising Diffusion Probabilistic Models. ICML 2021. arXiv:2102.09672. arxiv.org/abs/2102.09672
  2. Lin, S., Liu, B., Li, J. & Yang, X. (2023). Common Diffusion Noise Schedules and Sample Steps are Flawed. arXiv:2305.08891. arxiv.org/abs/2305.08891

Mentioned in

diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static., diffusion samplerdiffusion sampler, n.: The step-by-step routine a diffusion model follows to turn noise into a finished picture; the choice of sampler and number of steps trades speed against quality., denoising diffusion probabilistic modeldenoising diffusion probabilistic model, n.: The 2020 recipe that made diffusion image generators work well: add noise to pictures over a thousand small steps, then train a network to guess the noise so it can be removed., Jascha Sohl-DicksteinJascha Sohl-Dickstein, prop. n.: An American scientist who in 2015, with colleagues, introduced diffusion models, the technique behind most modern AI image and video generators, by borrowing ideas from physics.