Gymnasium
In plain wordsA free software library that gives reinforcement learning programs one standard way to plug into practice tasks, from simple games to simulated robots, so methods can be tried on identical problems. It succeeded OpenAI's Gym.
The Farama Foundation's open-source Python library for plugging reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time. code into the environmentsenvironment, n.: Everything an AI agent interacts with but does not control — a game, a simulator, a website, the physical world. It answers each of the agent's actions with a new situation and, in reinforcement learning, a score. it trains in: it fixes the programming interface every environment must present and bundles example environments of its own. An environment declares an observation space and an action spaceaction space, n.: The full menu of moves an AI agent is allowed to make — a handful of buttons in a video game, or any combination of motor forces for a robot arm. and provides reset(), which starts an episodeepisode, n.: One complete run of a task from start to finish — a single game of chess, one attempt at a maze — after which everything resets and the agent tries again. and returns the first observation, and step(action), which returns the next observation, the rewardreward, n.: The score an AI system receives after an action, telling it how good the outcome was. Reinforcement learning trains the system to collect as much reward as possible over time., separate terminated and truncated flags (distinguishing a true end of the task from a time limit) and diagnostic information. Bundled suites include classic control (CartPole, Pendulum), Box2D, toy-text and MuJoCoMuJoCo, prop. n.: A free physics simulator, now maintained by Google DeepMind, that AI researchers use to train and test simulated robots and bodies, especially with trial-and-error learning. physics tasks, and the Arcade Learning EnvironmentArcade Learning Environment, prop. n.: A software platform that lets AI programs play classic Atari 2600 video games, used since 2012 as a standard test of how well one AI can learn many different tasks from screen images. is available as a plugin; wrappers add preprocessing, recording and vectorized parallel execution. Because most RL libraries and many third-party environments implement this API, it serves as the field's common language, as OpenAI Gym did before it. Version 1.0 was released in October 2024.
Any environment that follows the Gymnasium API can be dropped into our PPO trainer unchanged.
Origin
OpenAI released Gym in April 2016, described in a June 2016 paper by Greg Brockman and colleagues. After OpenAI stopped active development, volunteers led by Jordan Terry took over maintenance in 2021 and in October 2022 continued the project as Gymnasium under the nonprofit Farama Foundation; Mark Towers and colleagues described it in a 2024 paper.
Usage note
Tutorials written for OpenAI Gym mostly work after replacing import gym with import gymnasium as gym, but older code expecting step() to return four values rather than five breaks.
Related
- environmentenvironment, n.: Everything an AI agent interacts with but does not control — a game, a simulator, a website, the physical world. It answers each of the agent's actions with a new situation and, in reinforcement learning, a score.
- Arcade Learning EnvironmentArcade Learning Environment, prop. n.: A software platform that lets AI programs play classic Atari 2600 video games, used since 2012 as a standard test of how well one AI can learn many different tasks from screen images.
- MuJoCoMuJoCo, prop. n.: A free physics simulator, now maintained by Google DeepMind, that AI researchers use to train and test simulated robots and bodies, especially with trial-and-error learning.
- RL environmentRL environment, n.: A practice world built for training an AI by trial and error: it hands the AI tasks and tools, lets it act, and scores the results so the AI can learn what works.
- benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time.
- action spaceaction space, n.: The full menu of moves an AI agent is allowed to make — a handful of buttons in a video game, or any combination of motor forces for a robot arm.
Sources
- Towers, M. et al. (2024). Gymnasium: A Standard Interface for Reinforcement Learning Environments. arXiv:2407.17032. arxiv.org/abs/2407.17032
- Brockman, G. et al. (2016). OpenAI Gym. arXiv:1606.01540. arxiv.org/abs/1606.01540
- Farama Foundation. Gymnasium (source repository and releases). github.com/Farama-Foundation/Gymnasium
Mentioned in
action spaceaction space, n.: The full menu of moves an AI agent is allowed to make — a handful of buttons in a video game, or any combination of motor forces for a robot arm., environmentenvironment, n.: Everything an AI agent interacts with but does not control — a game, a simulator, a website, the physical world. It answers each of the agent's actions with a new situation and, in reinforcement learning, a score., episodeepisode, n.: One complete run of a task from start to finish — a single game of chess, one attempt at a maze — after which everything resets and the agent tries again., physics simulatorphysics simulator, n.: Software that imitates how real objects move, collide, bend and fall, so robots and their AI can be designed, trained and tested on a computer before touching the real world., RL environmentRL environment, n.: A practice world built for training an AI by trial and error: it hands the AI tasks and tools, lets it act, and scores the results so the AI can learn what works., Arcade Learning EnvironmentArcade Learning Environment, prop. n.: A software platform that lets AI programs play classic Atari 2600 video games, used since 2012 as a standard test of how well one AI can learn many different tasks from screen images., Greg BrockmanGreg Brockman, prop. n.: An American engineer who helped start OpenAI in 2015 and became its president, after serving as the first technology chief of the payments company Stripe., MuJoCoMuJoCo, prop. n.: A free physics simulator, now maintained by Google DeepMind, that AI researchers use to train and test simulated robots and bodies, especially with trial-and-error learning.