few-shot prompting
In plain wordsShowing an AI a handful of examples of a task — inputs with the answers you want — inside the prompt, so it can copy the pattern for a new case.
Specifying a task by including a small number k of input–output demonstrations in the promptprompt, n.: The message or instructions you give an AI system — a question, a request, some examples or a document — that it responds to. Clearer prompts generally get better results., followed by a new input for the model to complete; one-shot prompting uses a single example, and many-shot prompting uses hundreds or thousands where the context windowcontext window, n.: How much an AI model can read and keep in view at once: the instructions, the conversation so far and any attached documents. Text that does not fit has to be cut, summarized or left out. allows. The model's weightsweights, n.: The learned numbers inside a neural network that set how strongly each part influences the next. The finished set of weights is, in effect, the trained model itself. do not change: adaptation happens through in-context learningin-context learning, n.: A language model's ability to pick up a new task from instructions or examples placed in its input, on the spot, without any retraining — like a quick study learning a game's rules from one round.. Demonstrations convey output format, label set and style more precisely than instructions alone, but results are sensitive to which examples are chosen, their order and the balance of labels (prompt sensitivityprompt sensitivity, n.: The way an AI's answers, and its test scores, can change a lot after small changes to a prompt that should not matter — a different word, an extra space or examples in another order.), and outputs can copy the examples' surface features. With strong instruction-tunedinstruction-tuned model, n.: A language model given extra training on examples of instructions paired with good responses, so it does what you ask instead of merely continuing your text. and reasoningreasoning model, n.: An AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning. models, carefully written zero-shot instructions often match few-shot prompts, but examples remain valuable for unusual formats and for base modelsbase model, n.: A language model fresh from its first broad training on text, before being taught to follow instructions or chat. It continues whatever text it is given instead of answering..
Two few-shot examples fixed the date format that the instructions alone never did.
Origin
The setting was defined in the GPT-3 paper by Tom Brown and colleagues at OpenAI (May 2020), which distinguished zero-, one- and few-shot settings, the last typically with 10 to 100 demonstrations; the term adapts 'few-shot learning' from meta-learning. Rishabh Agarwal and colleagues at Google DeepMind studied many-shot in-context learning with long-context models in 2024.
Usage note
Distinguish from classical few-shot learningfew-shot learning, n.: Getting an AI system to handle a new task or recognize a new kind of thing from just a handful of examples, the way a person can learn a new word after hearing it once or twice., which adapts a model's parameters (or a learned metric) from a few labeled examples.
Related
- in-context learningin-context learning, n.: A language model's ability to pick up a new task from instructions or examples placed in its input, on the spot, without any retraining — like a quick study learning a game's rules from one round.
- zero-shot learningzero-shot learning, n.: Doing a task without being shown a single example of it — say, an AI sorting reviews into positive and negative just from being told to, or recognizing an animal it never saw in training.
- prompt engineeringprompt engineering, n.: The skill of writing requests to an AI — picking the words, examples, format and background information it receives — so that it reliably gives the result you want, without retraining it.
- few-shot learningfew-shot learning, n.: Getting an AI system to handle a new task or recognize a new kind of thing from just a handful of examples, the way a person can learn a new word after hearing it once or twice.
- prompt sensitivityprompt sensitivity, n.: The way an AI's answers, and its test scores, can change a lot after small changes to a prompt that should not matter — a different word, an extra space or examples in another order.
- chain of thoughtchain of thought, n.: Getting an AI to write out its reasoning step by step before giving an answer, the way a student shows their working. This often makes answers to math and logic problems more accurate.
- zero-shot learningzero-shot learning, n.: Doing a task without being shown a single example of it — say, an AI sorting reviews into positive and negative just from being told to, or recognizing an animal it never saw in training.
- fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way.
Sources
- Brown, T. B. et al. (2020). Language Models are Few-Shot Learners. NeurIPS 2020. arxiv.org/abs/2005.14165v2
Mentioned in
few-shot learningfew-shot learning, n.: Getting an AI system to handle a new task or recognize a new kind of thing from just a handful of examples, the way a person can learn a new word after hearing it once or twice., promptprompt, n.: The message or instructions you give an AI system — a question, a request, some examples or a document — that it responds to. Clearer prompts generally get better results., prompt engineeringprompt engineering, n.: The skill of writing requests to an AI — picking the words, examples, format and background information it receives — so that it reliably gives the result you want, without retraining it., automated red teamingautomated red teaming, n.: Using AI programs, not just human testers, to invent huge numbers of tricky inputs that try to make another AI misbehave, so problems can be found and fixed at scale., base modelbase model, n.: A language model fresh from its first broad training on text, before being taught to follow instructions or chat. It continues whatever text it is given instead of answering., evaluation harnessevaluation harness, n.: Software that puts AI models through tests in a fixed, repeatable way — feeding in the questions, collecting the answers and scoring them — so that every model is judged identically., exampleexample, n.: A single item of data used to train or test an AI model, such as one photo with its label, one email marked spam or not, or one question with its answer., in-context learningin-context learning, n.: A language model's ability to pick up a new task from instructions or examples placed in its input, on the spot, without any retraining — like a quick study learning a game's rules from one round., MMLUMMLU, prop. n.: A giant multiple-choice exam for AI covering 57 subjects, from school math and history to law and medicine; for years it was the most quoted test of a model's general knowledge., one-shotone-shot, v.: Slang for an AI completing a whole task correctly on the first try from a single request — 'it one-shotted the app' — with no corrections or follow-up needed., prompt sensitivityprompt sensitivity, n.: The way an AI's answers, and its test scores, can change a lot after small changes to a prompt that should not matter — a different word, an extra space or examples in another order., prompt templateprompt template, n.: A reusable prompt with blanks to fill in — for example, 'Summarize this email for a busy manager: {email}' — so a program can send the same tested instructions with different content each time., stop sequencestop sequence, n.: A piece of text that tells an AI model to stop writing as soon as it produces it — for example, stopping at a blank line or just before the next question., zero-shot learningzero-shot learning, n.: Doing a task without being shown a single example of it — say, an AI sorting reviews into positive and negative just from being told to, or recognizing an animal it never saw in training., in-context vectorin-context vector, n.: A steering signal distilled from a few worked examples shown to a language model, so the model can follow the examples' pattern without having them repeated in every prompt., least-to-most promptingleast-to-most prompting, n.: A prompting method that has an AI list the smaller questions a hard problem depends on, answer the easiest first, and feed each answer into the next, building up to the full solution., many-shot jailbreakingmany-shot jailbreaking, n.: A way of breaking a chatbot's rules that exploits how much text it can read at once: the attacker pastes in hundreds of invented exchanges in which an AI obliges dangerous requests, and the model follows suit., prompt tuningprompt tuning, n.: Teaching a frozen language model a new task by learning a few made-up 'words' — lists of numbers rather than real text — that are placed before the input to steer the model.