seed AI
In plain wordsAn early AI built mainly to understand and upgrade itself, so that it could grow, like a seed, into a far more capable mind.
A hypothetical AI built above all to improve itself; in Eliezer Yudkowsky's original definition, one 'designed for self-understanding, self-modification, and recursive self-improvement' (see recursive self-improvementrecursive self-improvement, n.: An AI improving its own design or training, so that each improved version is better at making the next improvement. It is the engine usually imagined behind an intelligence explosion.). It might start out unimpressive, but by rewriting its own code and cognitive architecture it would climb to human level and beyond. The metaphor stresses that what matters is not the starting intelligence but the growth trajectory: the designers' main task would be to get the initial goal system right, because later versions would inherit and amplify it. This framing motivated early friendly AIfriendly AI, n.: An early name for AI built so that it reliably benefits people rather than harming them. The problem it named is now usually called AI alignment. research and the view that alignment must be solved before such a system is started. Modern discussion has largely replaced it with automated AI researchautomated AI research, n.: Using AI systems to do the work of AI researchers, such as running experiments and writing code for new models, so that AI helps build its own successors., in which AI systems improve their successors through ordinary training pipelines rather than by editing their own code.
The seed AI picture assumed hand-written code an AI could read and rewrite, not billions of learned weights.
Origin
Introduced by Eliezer Yudkowsky, by 2001, in writings for the Singularity Institute for Artificial Intelligence (now MIRIMachine Intelligence Research Institute, prop. n.: A small nonprofit in Berkeley, California, among the first to warn that smarter-than-human AI could destroy humanity, which now campaigns for a worldwide halt to building it.), notably 'General Intelligence and Seed AI' and 'Creating Friendly AI' (both 2001; the latter dated June 2001); an earlier first appearance is possible. Nick Bostrom adopted the term in Superintelligence (2014).
Related
- recursive self-improvementrecursive self-improvement, n.: An AI improving its own design or training, so that each improved version is better at making the next improvement. It is the engine usually imagined behind an intelligence explosion.
- intelligence explosionintelligence explosion, n.: The idea that once AI gets good enough at improving AI, each smarter version could build a smarter one faster still, so abilities would snowball far beyond human level in a short time.
- friendly AIfriendly AI, n.: An early name for AI built so that it reliably benefits people rather than harming them. The problem it named is now usually called AI alignment.
- Gödel machineGödel machine, n.: A purely theoretical design, proposed by Jürgen Schmidhuber, for a program allowed to change anything about itself, including how it learns, but only once it has proved that the change is an improvement.
- Eliezer YudkowskyEliezer Yudkowsky, prop. n.: An American writer and AI theorist who, from around 2000, argued that superhuman AI would likely destroy humanity unless its goals were made safe, and who founded an online community devoted to rational thinking.
- automated AI researchautomated AI research, n.: Using AI systems to do the work of AI researchers, such as running experiments and writing code for new models, so that AI helps build its own successors.
Mentioned in
automated AI researchautomated AI research, n.: Using AI systems to do the work of AI researchers, such as running experiments and writing code for new models, so that AI helps build its own successors., recursive self-improvementrecursive self-improvement, n.: An AI improving its own design or training, so that each improved version is better at making the next improvement. It is the engine usually imagined behind an intelligence explosion., friendly AIfriendly AI, n.: An early name for AI built so that it reliably benefits people rather than harming them. The problem it named is now usually called AI alignment., Gödel machineGödel machine, n.: A purely theoretical design, proposed by Jürgen Schmidhuber, for a program allowed to change anything about itself, including how it learns, but only once it has proved that the change is an improvement.