environment
In plain wordsEverything an AI agent interacts with but does not control — a game, a simulator, a website, the physical world. It answers each of the agent's actions with a new situation and, in reinforcement learning, a score.
- 1
Reinforcement learning
The part of a reinforcement-learning problem outside the agentagent, n.: Something that acts. In AI, a program that observes its situation and takes actions toward a goal: a game-playing learner, a software helper, or a chatbot that uses tools to complete tasks.'s control: the system that receives each actionaction, n.: A choice an AI agent makes at one moment — move left, place a stone, type a command — which changes its situation and may earn it a reward., changes its statestate, n.: A snapshot of the situation an AI agent is in — for a chess program, the arrangement of pieces and whose turn it is — containing what it needs to decide what to do next. according to its own dynamics, and emits the next observation and rewardreward, n.: The score an AI system receives after an action, telling it how good the outcome was. Reinforcement learning trains the system to collect as much reward as possible over time.. Formally it is specified by the transition and reward functions of a Markov decision processMarkov decision process, n.: A mathematical description of a step-by-step decision problem: each step you pick an action, receive a reward and then land in a new situation, drawn at random from odds that depend only on the current situation and your choice., or of a partially observable Markov decision processpartially observable Markov decision process, n.: A version of the standard decision-making model in which the decision-maker cannot see the whole situation — like a poker player who cannot see other hands — and must act on clues and memory. when the agent sees only observations. Following Richard Sutton and Andrew Barto's convention, the boundary marks the limit of what the agent directly controls rather than the edge of its body, so a robot's motors, and even the computation of its reward, are usually treated as part of the environment. In software, an environment is a program with a standard interface — typically reset() to begin an episodeepisode, n.: One complete run of a task from start to finish — a single game of chess, one attempt at a maze — after which everything resets and the agent tries again. and step(action) to advance it — as popularized by OpenAI Gym and its successor GymnasiumGymnasium, prop. n.: A free software library that gives reinforcement learning programs one standard way to plug into practice tasks, from simple games to simulated robots, so methods can be tried on identical problems. It succeeded OpenAI's Gym..
The environment returns a terminated flag when the pole falls and a truncated flag after 500 steps.
- 2
Agents and post-training
A packaged task setting in which a large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. or AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done. acts — a code repository with tests, a web browser, a simulated customer — together with the tools it may call and a grader that scores the outcome. Built for training or evaluating agents, such settings are treated under RL environmentRL environment, n.: A practice world built for training an AI by trial and error: it hands the AI tasks and tools, lets it act, and scores the results so the AI can learn what works..
Origin
The framing comes from control engineering, where the controlled system is called the 'plant', and from cybernetics. It became standard through AI textbooks that define agents by the environments they perceive and act on (Stuart Russell and Peter Norvig, 1995) and through Sutton and Barto's RL textbook (1998). Software such as RL-Glue (2009) and, above all, OpenAI Gym (2016) made 'environment' the name of a program with a standard interface.
Usage note
Environments are usually assumed fixed while the agent learns; in multi-agent reinforcement learningmulti-agent reinforcement learning, n.: Reinforcement learning with several AI agents learning at once in a shared world, where each must adapt to the others — as teammates, rivals or both. the other learning agents are part of each agent's environment, which therefore keeps changing.
Related
- agentagent, n.: Something that acts. In AI, a program that observes its situation and takes actions toward a goal: a game-playing learner, a software helper, or a chatbot that uses tools to complete tasks.
- Markov decision processMarkov decision process, n.: A mathematical description of a step-by-step decision problem: each step you pick an action, receive a reward and then land in a new situation, drawn at random from odds that depend only on the current situation and your choice.
- statestate, n.: A snapshot of the situation an AI agent is in — for a chess program, the arrangement of pieces and whose turn it is — containing what it needs to decide what to do next.
- actionaction, n.: A choice an AI agent makes at one moment — move left, place a stone, type a command — which changes its situation and may earn it a reward.
- rewardreward, n.: The score an AI system receives after an action, telling it how good the outcome was. Reinforcement learning trains the system to collect as much reward as possible over time.
- GymnasiumGymnasium, prop. n.: A free software library that gives reinforcement learning programs one standard way to plug into practice tasks, from simple games to simulated robots, so methods can be tried on identical problems. It succeeded OpenAI's Gym.
- RL environmentRL environment, n.: A practice world built for training an AI by trial and error: it hands the AI tasks and tools, lets it act, and scores the results so the AI can learn what works.
- physics simulatorphysics simulator, n.: Software that imitates how real objects move, collide, bend and fall, so robots and their AI can be designed, trained and tested on a computer before touching the real world.
- agentagent, n.: Something that acts. In AI, a program that observes its situation and takes actions toward a goal: a game-playing learner, a software helper, or a chatbot that uses tools to complete tasks.
Sources
- Brockman, G. et al. (2016). OpenAI Gym. arXiv:1606.01540. arxiv.org/abs/1606.01540
Mentioned in
reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time., rewardreward, n.: The score an AI system receives after an action, telling it how good the outcome was. Reinforcement learning trains the system to collect as much reward as possible over time., world modelworld model, n.: An AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real., agentagent, n.: Something that acts. In AI, a program that observes its situation and takes actions toward a goal: a game-playing learner, a software helper, or a chatbot that uses tools to complete tasks., episodeepisode, n.: One complete run of a task from start to finish — a single game of chess, one attempt at a maze — after which everything resets and the agent tries again., GymnasiumGymnasium, prop. n.: A free software library that gives reinforcement learning programs one standard way to plug into practice tasks, from simple games to simulated robots, so methods can be tried on identical problems. It succeeded OpenAI's Gym., intelligent agentintelligent agent, n.: In AI theory, a decision-maker, such as a robot, a self-driving car, a chess program or a helpful piece of software, that notices what is going on around it and responds in ways that serve its goals., model-based reinforcement learningmodel-based reinforcement learning, n.: Learning in which an AI agent builds, or is given, an inner simulation of how its world responds, and uses it to plan or practice in imagination before acting., model-free reinforcement learningmodel-free reinforcement learning, n.: Learning to act well purely from trial and error, without ever building an internal picture of how the world works — like learning a game by habit rather than by thinking moves ahead., multi-agent reinforcement learningmulti-agent reinforcement learning, n.: Reinforcement learning with several AI agents learning at once in a shared world, where each must adapt to the others — as teammates, rivals or both., RL environmentRL environment, n.: A practice world built for training an AI by trial and error: it hands the AI tasks and tools, lets it act, and scores the results so the AI can learn what works., rolloutrollout, n.: Letting an AI play out a task with its current strategy, from some starting point to the end, to see how things turn out and gather data for learning or planning., statestate, n.: A snapshot of the situation an AI agent is in — for a chess program, the arrangement of pieces and whose turn it is — containing what it needs to decide what to do next., training steptraining step, n.: One round of learning: the model makes predictions on a small batch of examples, measures its error and adjusts its settings once. Training runs consist of thousands to millions of steps., trajectorytrajectory, n.: The record of what happened as an AI agent acted: the sequence of situations it was in, the actions it took and the rewards it received., A3CA3C, abbr.: A 2016 DeepMind training method in which many copies of an AI agent learn in parallel on separate copies of a task, pooling their updates into one shared neural network., Arcade Learning EnvironmentArcade Learning Environment, prop. n.: A software platform that lets AI programs play classic Atari 2600 video games, used since 2012 as a standard test of how well one AI can learn many different tasks from screen images., DynaDyna, prop. n.: An early design in which a learning agent builds a model of its world from real experience, then also practices on imagined experience from that model, squeezing more learning out of each real step., inverse reinforcement learninginverse reinforcement learning, n.: Working backward from behavior to goals: watching an expert act and inferring what they must be trying to achieve, so an AI can pursue the same aims., offline reinforcement learningoffline reinforcement learning, n.: Learning a strategy entirely from a fixed collection of recorded experience — logs, past robot runs, medical records — without trying anything new in the real world while learning., reward is enoughreward is enough, phr.: The claim, made by DeepMind researchers in 2021, that maximizing a single reward in a rich enough world could by itself produce all the abilities we associate with intelligence.