Skip to content
Last Dictionary
Plane
Data & DatasetsTraining & Post-trainingLanguage Models & Transformers

Self-Instruct

nounAlso: Self-Instruct method, self-generated instructions

In plain wordsA technique in which a language model writes its own practice tasks and answers, starting from a handful of human examples; the results are filtered and used to train it to follow instructions.

  1. A way of having a language model write the training data for its own instruction tuninginstruction tuning, n.: Training a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text., growing an instruction datasetinstruction dataset, n.: A collection of example requests paired with good answers, used to teach a pretrained language model to follow instructions and act as a helpful assistant. in repeated rounds. Starting from a small pool of human-written seed tasks (175 in the original work), the model is prompted with sampled examples to propose new instructions, decide whether each is a classification task, and generate inputs and outputs; heuristics then discard invalid outputs and instructions too similar to existing ones (measured by ROUGE-L overlap), and the survivors join the pool for further rounds. In the original experiment, about 52,000 instructions generated this way lifted the original GPT-3 by 33 percentage points on the Super-NaturalInstructions benchmark, bringing it roughly level with OpenAI's instruction-tuned InstructGPT-001. Stanford's Alpaca (2023) applied a simplified version with a stronger model to fine-tune LLaMA cheaply, starting a wave of open models trained on model-generated synthetic datasynthetic data, n.: Data created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars..

    We ran Self-Instruct from 200 seed tasks in our domain and filtered aggressively for near-duplicates.

Origin

Introduced by Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi and Hannaneh Hajishirzi in 'Self-Instruct: Aligning Language Models with Self-Generated Instructions' (arXiv, December 2022; ACL 2023).

Usage note

Generated data inherits the generator's errors and biases and can lack diversity; later methods such as Evol-Instruct made instructions progressively harder, and pipelines added stronger filtering. When the generating model belongs to someone else, its terms of use may restrict the practice.

Related

See also
  • instruction datasetinstruction dataset, n.: A collection of example requests paired with good answers, used to teach a pretrained language model to follow instructions and act as a helpful assistant.
  • synthetic datasynthetic data, n.: Data created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars.
  • instruction tuninginstruction tuning, n.: Training a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text.
  • knowledge distillationknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run.
  • self-improvementself-improvement, n.: When an AI gets better by learning from material it produces itself — its own answers, critiques or practice problems — rather than only from new human-made examples.

Sources

  1. Wang, Y. et al. (2023). Self-Instruct: Aligning Language Models with Self-Generated Instructions. ACL 2023. arXiv:2212.10560. arxiv.org/abs/2212.10560

Mentioned in

synthetic datasynthetic data, n.: Data created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars., instruction datasetinstruction dataset, n.: A collection of example requests paired with good answers, used to teach a pretrained language model to follow instructions and act as a helpful assistant., instruction tuninginstruction tuning, n.: Training a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text.