Skip to content
Last Dictionary
Plane
AGI & SuperintelligenceMathematics & StatisticsReinforcement Learning

AIXI

proper nounAlso: AIXItl

In plain wordsA mathematical description of a perfect learning agent that weighs every possible explanation of its experience and acts to get the most future reward. It cannot actually be built, but it defines an ideal.

  1. A theoretical reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time. agent, defined by Marcus Hutter, that is optimal in a precise sense across all computable environments. It predicts with Solomonoff inductionSolomonoff induction, n.: A mathematical recipe for ideal prediction: consider every computer program that could have produced the data so far, trust shorter programs more, and predict what they would output next., giving every program that could have produced its observations so far a prior weight of 2 to the power of minus its length, so that simpler explanations dominate, and it plans with that prediction: at each step it picks the action whose expected total reward up to a fixed horizon is highest under the resulting mixture. Because it sums over all programs, AIXI is incomputable; time- and length-bounded variants (AIXItl) are computable but intractable, and Monte Carlo approximations have learned to play simple games. AIXI serves as a reference point for theoretical analyses of general intelligence and of safety problems such as wireheadingwireheading, n.: When an AI, animal or person stimulates its own reward signal directly instead of doing anything useful — like a rat pressing a lever wired to its brain's pleasure center. and self-modification. Criticisms include that its optimality depends on the choice of reference universal Turing machine, and that it treats itself as separate from its environment, so it cannot reason about being modified or destroyed (embedded agencyembedded agency, n.: The puzzle of how an agent should reason when it is itself part of the world it is trying to understand and change, rather than standing outside it.).

    AIXI tells you what a perfect reward maximizer would do, which is exactly why safety researchers study its failure modes.

ak:=arg⁡max⁡ak∑okrk⋯max⁡am∑omrm[rk+⋯+rm]∑q : U(q, a1…am)=o1r1…omrm2−ℓ(q)a_k := \arg\max_{a_k} \sum_{o_k r_k} \cdots \max_{a_m} \sum_{o_m r_m} \big[ r_k + \cdots + r_m \big] \sum_{q \,:\, U(q,\, a_1 \ldots a_m) = o_1 r_1 \ldots o_m r_m} 2^{-\ell(q)}

Origin

Introduced by Marcus Hutter in a 2000 paper, 'A Theory of Universal Artificial Intelligence based on Algorithmic Complexity', and developed in his book Universal Artificial Intelligence (2005). The name joins AI with ξ (xi), Hutter's symbol for the universal mixture distribution. Joel Veness and colleagues published a Monte Carlo approximation in 2011.

Related

See also
  • universal intelligenceuniversal intelligence, n.: A mathematical definition of intelligence as an agent's average ability to achieve goals across every possible environment, with simpler environments counting more.
  • Solomonoff inductionSolomonoff induction, n.: A mathematical recipe for ideal prediction: consider every computer program that could have produced the data so far, trust shorter programs more, and predict what they would output next.
  • Kolmogorov complexityKolmogorov complexity, n.: How briefly something can be described by a computer program that recreates it exactly: a million zeros needs only a tiny program, while a truly random string needs one about as long as itself.
  • reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.
  • embedded agencyembedded agency, n.: The puzzle of how an agent should reason when it is itself part of the world it is trying to understand and change, rather than standing outside it.
  • Shane LeggShane Legg, prop. n.: A New Zealand-born AI researcher who co-founded DeepMind and helped make 'artificial general intelligence' a mainstream term and goal.

Sources

  1. Hutter, M. (2005). Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Springer.
  2. Veness, J., Ng, K. S., Hutter, M., Uther, W. & Silver, D. (2011). A Monte-Carlo AIXI Approximation. Journal of Artificial Intelligence Research 40.
  3. Leike, J. & Hutter, M. (2015). Bad Universal Priors and Notions of Optimality. Proceedings of COLT 2015.

Mentioned in

bounded rationalitybounded rationality, n.: The idea that real decision-makers, whether people or machines, cannot consider every option or compute the perfect choice, so they make reasonable decisions within limits of time, information and thinking power., Shane LeggShane Legg, prop. n.: A New Zealand-born AI researcher who co-founded DeepMind and helped make 'artificial general intelligence' a mainstream term and goal., agent foundationsagent foundations, n.: A research program that tried to work out, with mathematics, how an idealized reasoning agent should think and decide, as groundwork for building AI that stays safe., embedded agencyembedded agency, n.: The puzzle of how an agent should reason when it is itself part of the world it is trying to understand and change, rather than standing outside it., Gödel machineGödel machine, n.: A purely theoretical design, proposed by Jürgen Schmidhuber, for a program allowed to change anything about itself, including how it learns, but only once it has proved that the change is an improvement., Kolmogorov complexityKolmogorov complexity, n.: How briefly something can be described by a computer program that recreates it exactly: a million zeros needs only a tiny program, while a truly random string needs one about as long as itself., Marcus HutterMarcus Hutter, prop. n.: A German computer scientist who defined AIXI, a mathematical ideal of a perfectly intelligent agent, and who argues that compressing data well is a measure of intelligence., Ray SolomonoffRay Solomonoff, prop. n.: An American mathematician who worked out an ideal mathematical recipe for predicting what comes next from past data, and who attended the 1956 meeting that launched AI., Solomonoff inductionSolomonoff induction, n.: A mathematical recipe for ideal prediction: consider every computer program that could have produced the data so far, trust shorter programs more, and predict what they would output next., universal intelligenceuniversal intelligence, n.: A mathematical definition of intelligence as an agent's average ability to achieve goals across every possible environment, with simpler environments counting more.