Skip to content
Last Dictionary

DreamBooth

proper nounAlso: subject-driven generation, prior-preservation loss

In plain wordsA way to teach an image generator one specific subject — your dog, your face, a product — from a handful of photos, so it can draw that exact subject in new scenes.

  1. A fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way. method that teaches a text-to-image modeltext-to-image model, n.: An AI system that creates a picture from a written description, such as 'a fox reading a newspaper, in watercolor', usually within seconds. one particular subject — a certain dog, face or product — from typically three to five photos, after which prompts such as 'a [V] dog on the moon' render that subject faithfully in new settings, poses and styles. The photos are paired with prompts in which a rarely used token, here [V], serves as the subject's name beside its class word ('a [V] dog'), and the whole model is updated. So that the model neither forgets what ordinary members of the class look like nor turns every dog into the subject (language drift), a class-specific prior-preservation loss also trains it on images it generated beforehand from the plain class prompt. Full DreamBooth produces a multi-gigabyte model per subject; community practice soon combined it with LoRAlow-rank adaptation, n.: A cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves. to produce small add-on files. Because it can depict real people realistically from a few photos, it raised immediate concerns about non-consensual imagery and deepfakesdeepfake, n.: A fake but realistic video, photo or voice recording made with AI that puts words in a real person's mouth or shows them in a scene that never happened..

Origin

Introduced by Nataniel Ruiz and colleagues at Google Research and Boston University in a paper posted in August 2022 (CVPR 2023), demonstrated on Google's Imagen; open-source implementations for Stable Diffusion appeared within weeks.

Related

See also
  • textual inversiontextual inversion, n.: Teaching an image generator a new 'word' for a specific object or style from a few example pictures, leaving the model itself unchanged — only one vocabulary entry is added.
  • low-rank adaptationlow-rank adaptation, n.: A cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves.
  • fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way.
  • text-to-image modeltext-to-image model, n.: An AI system that creates a picture from a written description, such as 'a fox reading a newspaper, in watercolor', usually within seconds.
  • Stable DiffusionStable Diffusion, prop. n.: A widely used image-generating AI whose model files were released for anyone to download, so people could run it on their own computers and adapt it.
  • deepfakedeepfake, n.: A fake but realistic video, photo or voice recording made with AI that puts words in a real person's mouth or shows them in a scene that never happened.
Contrast with
  • textual inversiontextual inversion, n.: Teaching an image generator a new 'word' for a specific object or style from a few example pictures, leaving the model itself unchanged — only one vocabulary entry is added.

Sources

  1. Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M. & Aberman, K. (2023). DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation. CVPR 2023. arXiv:2208.12242. arxiv.org/abs/2208.12242

Mentioned in

Stable DiffusionStable Diffusion, prop. n.: A widely used image-generating AI whose model files were released for anyone to download, so people could run it on their own computers and adapt it., ControlNetControlNet, prop. n.: An add-on for image generators that makes them follow a guide image — a sketch, a stick-figure pose or a depth map — so the layout comes out as intended., textual inversiontextual inversion, n.: Teaching an image generator a new 'word' for a specific object or style from a few example pictures, leaving the model itself unchanged — only one vocabulary entry is added.