text-to-image model
In plain wordsAn AI system that creates a picture from a written description, such as 'a fox reading a newspaper, in watercolor', usually within seconds.
A generative modelgenerative model, n.: A model that learns what a kind of data looks like — faces, sentences, songs — well enough to produce new examples that resemble the originals without copying them. that learns from hundreds of millions to billions of captioned images, increasingly recaptioned in detail by vision-language models, to produce new images that match a natural-language promptprompt, n.: The message or instructions you give an AI system — a question, a request, some examples or a document — that it responds to. Clearer prompts generally get better results.. In the usual design a text encoder (a frozen language model such as T5, a CLIPCLIP, prop. n.: An OpenAI model that learned to match pictures with their captions from hundreds of millions of web examples, so it can judge how well any piece of text describes any image. text encoder or a full large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code.) first turns the prompt into vectors, and an image generator conditioned on them, today usually a latent diffusionlatent diffusion model, n.: A diffusion image generator that works on a shrunken, compressed version of each picture instead of the full-size pixels, which makes it far cheaper to train and run. or flow matchingflow matching, n.: A way to train generators that learns, for every point between random noise and real data, which direction to move, so that noise can be steered smoothly into new examples. transformerdiffusion transformer, n.: A diffusion image or video generator whose core network is a transformer — the architecture behind chatbots — instead of the older image-specific design, which lets it improve more predictably as it grows. steered with classifier-free guidanceclassifier-free guidance, n.: A setting that makes an image generator follow the prompt more strictly by comparing what it would draw with and without the prompt, then exaggerating the difference., builds the picture; other models emit image tokenstoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens. autoregressivelyautoregressive image generation, n.: Making a picture one piece at a time, each piece predicted from those already made — the way chatbots write text word by word., and natively multimodal models produce images inside a chat model. Typical controls include the prompt, negative promptnegative prompt, n.: A second prompt listing what you do not want in a generated image — 'blurry, extra fingers, watermark' — which the generator then steers away from., random seedrandom seed, n.: A number that fixes the 'random' choices a computer makes; reuse the same seed and settings, and an image generator produces the same picture again., aspect ratio, reference images and structural conditions such as ControlNetControlNet, prop. n.: An add-on for image generators that makes them follow a guide image — a sketch, a stick-figure pose or a depth map — so the layout comes out as intended.; most systems also support inpaintinginpainting, n.: Filling in a missing or erased part of a picture so it blends seamlessly with the rest — removing a stranger from a photo, or repairing a damaged one. and instruction-based image editinginstruction-based image editing, n.: Changing a picture by telling an AI what to change — 'make it night', 'remove the car', 'give her a red coat' — instead of using editing tools by hand.. Quality is judged by human preference ratings and arena leaderboards and by metrics such as FIDFréchet inception distance, n.: A score for image generators that compares the overall statistics of their pictures with those of real photos, as seen by an image-recognition network; lower is better. and CLIP scoreCLIP score, n.: A score for how well a picture matches a text description, computed by an AI model that was trained to pair images with their captions.. Long-standing weaknesses — counting, spatial relations, legible text, hands — narrowed sharply between 2022 and 2026. Training on scraped artwork and photographs is the subject of copyright lawsuitsAI copyright litigation, n.: The many lawsuits in which authors, artists, news organizations, record labels and others accuse AI companies of copying their work without permission to build or run AI models., and outputs raise concerns about deepfakesdeepfake, n.: A fake but realistic video, photo or voice recording made with AI that puts words in a real person's mouth or shows them in a scene that never happened., bias and provenance.
The text-to-image model nailed the composition but misspelled the shop sign.
Origin
Early neural text-to-image systems include alignDRAW (Elman Mansimov and colleagues, posted November 2015) and GAN-based models by Scott Reed and colleagues (2016). OpenAI's DALL·E (January 2021) showed broad compositional ability, and in 2022 DALL·E 2, Imagen, Midjourney and Stable Diffusion brought the technology to the public.
Related
- diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static.
- DALL·EDALL·E, prop. n.: OpenAI's pioneering series of image generators, named after the painter Salvador Dalí and the robot WALL·E, which showed in 2021–2023 that AI could draw almost anything described in words.
- Stable DiffusionStable Diffusion, prop. n.: A widely used image-generating AI whose model files were released for anyone to download, so people could run it on their own computers and adapt it.
- MidjourneyMidjourney, prop. n.: A popular paid AI image generator, first used by typing prompts into the chat app Discord, known for its polished, painterly look; it can also animate images into short videos.
- ImagenImagen, prop. n.: Google's family of AI image generators, first shown in 2022, whose research showed that better understanding of language yields pictures that match descriptions more faithfully.
- text-to-video modeltext-to-video model, n.: An AI that creates a short video clip — increasingly with matching sound — from a written description, such as 'a paper boat sailing down a rainy street'.
- AI artAI art, n.: Images, music or other artworks made with the help of AI — from early computer drawing programs to today's prompt-based image generators — and the debate over whether and how such work counts as art.
- promptprompt, n.: The message or instructions you give an AI system — a question, a request, some examples or a document — that it responds to. Clearer prompts generally get better results.
- image captioningimage captioning, n.: Having a computer write a short description of a picture, such as 'a dog catching a frisbee in a park', to help blind people, search engines and other AI systems.
Sources
- Ramesh, A. et al. (2021). Zero-Shot Text-to-Image Generation. ICML 2021. arXiv:2102.12092. arxiv.org/pdf/2102.12092v1.pdf
- Mansimov, E., Parisotto, E., Ba, J. L. & Salakhutdinov, R. (2016). Generating Images from Captions with Attention. ICLR 2016.
- Reed, S. et al. (2016). Generative Adversarial Text to Image Synthesis. ICML 2016.
Mentioned in
AI artAI art, n.: Images, music or other artworks made with the help of AI — from early computer drawing programs to today's prompt-based image generators — and the debate over whether and how such work counts as art., generative AIgenerative AI, n.: AI that produces something new on request, such as an essay, a picture, a song or a piece of code, having absorbed from enormous numbers of existing examples what such things typically look like., MidjourneyMidjourney, prop. n.: A popular paid AI image generator, first used by typing prompts into the chat app Discord, known for its polished, painterly look; it can also animate images into short videos., promptprompt, n.: The message or instructions you give an AI system — a question, a request, some examples or a document — that it responds to. Clearer prompts generally get better results., Stable DiffusionStable Diffusion, prop. n.: A widely used image-generating AI whose model files were released for anyone to download, so people could run it on their own computers and adapt it., text-to-video modeltext-to-video model, n.: An AI that creates a short video clip — increasingly with matching sound — from a written description, such as 'a paper boat sailing down a rainy street'., classifier-free guidanceclassifier-free guidance, n.: A setting that makes an image generator follow the prompt more strictly by comparing what it would draw with and without the prompt, then exaggerating the difference., CLIPCLIP, prop. n.: An OpenAI model that learned to match pictures with their captions from hundreds of millions of web examples, so it can judge how well any piece of text describes any image., ControlNetControlNet, prop. n.: An add-on for image generators that makes them follow a guide image — a sketch, a stick-figure pose or a depth map — so the layout comes out as intended., cross-attentioncross-attention, n.: Attention between two different inputs: one sequence consults another, as when a translation model writing French looks back at the English sentence, or an image generator consults the text prompt., DALL·EDALL·E, prop. n.: OpenAI's pioneering series of image generators, named after the painter Salvador Dalí and the robot WALL·E, which showed in 2021–2023 that AI could draw almost anything described in words., DreamBoothDreamBooth, prop. n.: A way to teach an image generator one specific subject — your dog, your face, a product — from a handful of photos, so it can draw that exact subject in new scenes., FLUXFLUX, prop. n.: A family of image-generating AI models from the German start-up Black Forest Labs, founded by researchers behind Stable Diffusion; some versions can be downloaded and run freely., generative artgenerative art, n.: Art in which the artist invents a process — dice rolls, software, written instructions — and lets that process make the work; the phrase now also loosely covers pictures from AI image generators., image captioningimage captioning, n.: Having a computer write a short description of a picture, such as 'a dog catching a frisbee in a park', to help blind people, search engines and other AI systems., ImagenImagen, prop. n.: Google's family of AI image generators, first shown in 2022, whose research showed that better understanding of language yields pictures that match descriptions more faithfully., inpaintinginpainting, n.: Filling in a missing or erased part of a picture so it blends seamlessly with the rest — removing a stranger from a photo, or repairing a damaged one., instruction-based image editinginstruction-based image editing, n.: Changing a picture by telling an AI what to change — 'make it night', 'remove the car', 'give her a red coat' — instead of using editing tools by hand., multimodal AImultimodal AI, n.: AI that can take in or produce more than one kind of information, such as text, images, sound and video, so you can show it a photo and ask questions about it., Nano BananaNano Banana, prop. n.: The nickname, later Google's official branding, for its Gemini image models, a viral hit in 2025 for editing photos from simple instructions while keeping people looking like themselves., negative promptnegative prompt, n.: A second prompt listing what you do not want in a generated image — 'blurry, extra fingers, watermark' — which the generator then steers away from., AI handsAI hands, n.: The tendency of early AI image generators to draw hands badly — extra or missing fingers, twisted joints — which became a running joke and a quick way to spot fake images., Balenciaga PopeBalenciaga Pope, prop. n.: A 2023 AI-generated photo of Pope Francis in a stylish white puffer coat that fooled many viewers, becoming a landmark example of how convincing fake images had become., Black Forest LabsBlack Forest Labs, prop. n.: A German AI company founded in 2024 by original creators of Stable Diffusion, which makes the FLUX image-generation models.