robot learning
In plain wordsTeaching robots skills from data and experience, such as watching demonstrations, practicing by trial and error or training in simulation, instead of programming every movement by hand.
The application of machine learningmachine learning, n.: A way of building software that learns from examples instead of following rules written by a programmer. Shown thousands of labeled photos, for instance, a program can learn to recognize cats. to give robotsrobot, n.: A machine that senses its surroundings, decides what to do and physically acts, moving itself or handling objects, with some independence from a human operator. Factory arms, robot vacuums and self-driving cars are all robots. perception, control and decision-making skills that would be impractical to program by hand. The main approaches are imitation learningimitation learning, n.: Teaching an AI by showing it examples of an expert doing a task, so it learns to copy the behavior instead of discovering it through trial and error. from human demonstrations, usually collected by teleoperationteleoperation, n.: Controlling a robot or machine from a distance, with a person steering it in real time using a joystick, a matching 'puppet' arm, a VR headset or a screen. (behavioral cloningbehavioral cloning, n.: The simplest way to copy an expert: record what the expert did in each situation and train an AI to predict the same action whenever it sees a similar situation., diffusion policiesdiffusion policy, n.: A way of training robots in which the same kind of AI that generates images instead generates the robot's next movements, letting it copy demonstrations even when there are several good ways to do a task., action chunkingaction chunking, n.: Having a robot's AI plan a short burst of upcoming movements, say the next second, all at once instead of one tiny motion at a time, which makes motion smoother and mistakes rarer.); reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time., typically in a physics simulatorphysics simulator, n.: Software that imitates how real objects move, collide, bend and fall, so robots and their AI can be designed, trained and tested on a computer before touching the real world. followed by sim-to-real transfersim-to-real transfer, n.: Training a robot's AI in a computer simulation, where practice is fast, cheap and safe, and then making it work on the real robot despite the differences between simulation and reality.; self-supervised learning from the robot's own interaction data, as in large-scale grasping experiments; and, since 2023, robot foundation modelsrobot foundation model, n.: One large AI model trained on huge amounts of robot and internet data so that it can control many robots on many tasks, and be adapted quickly to new ones. such as vision-language-action modelsvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements., pretrained on web data plus multi-robot datasets like Open X-EmbodimentOpen X-Embodiment, prop. n.: A large shared collection of robot training data, pooled by dozens of research labs in 2023 and covering many kinds of robots, so that one AI model could learn from all of them.. Compared with other machine learning, its distinctive difficulties are the cost and slowness of real-world data, safety during exploration, partial observability, distribution shiftdistribution shift, n.: When the data an AI system meets in real use differs from the data it was trained on — new users, places, times or habits — often making it perform worse than tests suggested. that compounds over long action sequences, and transfer across different bodies (cross-embodiment learningcross-embodiment learning, n.: Training one robot AI on data from many differently shaped robots, such as arms, robot dogs, drones and humanoids, so that skills learned on one body help another.).
Robot learning papers now report hours of teleoperated data the way language-model papers report tokens.
Origin
Machine learning for robot control dates to the 1980s; the name was in use by at least 1993, when Jonathan Connell and Sridhar Mahadevan's edited volume Robot Learning appeared. Large-scale deep-learning studies, notably Sergey Levine and colleagues' 2016 grasping experiments with 14 robot arms and Google's QT-Opt (2018), and the first Conference on Robot Learning (CoRL) in 2017 made it a distinct research community.
Usage note
Most deployed robots still combine learned components (perception, grasp selection) with classical planning and control; fully end-to-end learned control is common in research and increasingly in products.
Related
- imitation learningimitation learning, n.: Teaching an AI by showing it examples of an expert doing a task, so it learns to copy the behavior instead of discovering it through trial and error.
- reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.
- sim-to-real transfersim-to-real transfer, n.: Training a robot's AI in a computer simulation, where practice is fast, cheap and safe, and then making it work on the real robot despite the differences between simulation and reality.
- robot foundation modelrobot foundation model, n.: One large AI model trained on huge amounts of robot and internet data so that it can control many robots on many tasks, and be adapted quickly to new ones.
- vision-language-action modelvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements.
- teleoperationteleoperation, n.: Controlling a robot or machine from a distance, with a person steering it in real time using a joystick, a matching 'puppet' arm, a VR headset or a screen.
- embodied AIembodied AI, n.: AI that has a body, usually a robot and sometimes a simulated character, and learns or acts by sensing and moving in a world rather than only processing text or images.
- control theorycontrol theory, n.: The mathematics of making machines behave as intended by constantly measuring what they are doing and correcting them, as a thermostat, cruise control or self-balancing robot does.
Sources
- Connell, J. H. & Mahadevan, S. (eds.) (1993). Robot Learning. Kluwer Academic.
- Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J. & Quillen, D. (2018). Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. International Journal of Robotics Research 37(4–5).
Mentioned in
roboticsrobotics, n.: The science and engineering of building robots and making them work: designing their bodies, sensors and motors, and writing the software that lets them see, plan and move safely., embodied AIembodied AI, n.: AI that has a body, usually a robot and sometimes a simulated character, and learns or acts by sensing and moving in a world rather than only processing text or images., graspinggrasping, n.: How a robot picks up and holds an object securely with a gripper or hand: deciding where to put its fingers or suction cup and how hard to squeeze., Moravec's paradoxMoravec's paradox, n.: The observation that what is hard for people, like chess or algebra, is often easy for computers, while what toddlers do effortlessly, such as seeing, walking and picking things up, is very hard for machines., teleoperationteleoperation, n.: Controlling a robot or machine from a distance, with a person steering it in real time using a joystick, a matching 'puppet' arm, a VR headset or a screen., Chelsea FinnChelsea Finn, prop. n.: An American computer scientist at Stanford known for 'learning to learn' methods that let AI adapt to new tasks from a few examples, and a co-founder of the robotics company Physical Intelligence., CoRLCoRL, prop. n.: A yearly academic conference, started in 2017, for research where robotics meets machine learning., DAggerDAgger, n.: A way of teaching a robot by imitation in which an expert keeps labeling what it should have done in the situations it actually gets into, so it learns to recover from its own mistakes., generalist robot policygeneralist robot policy, n.: A single robot-control program trained to do many different tasks in many places, often on several kinds of robots: the opposite of a specialist trained for one job., goal-conditioned reinforcement learninggoal-conditioned reinforcement learning, n.: Training one AI agent to reach many different goals — whichever target it is given — instead of a single fixed task, so a robot can, say, place an object anywhere asked., ICRAICRA, prop. n.: The largest yearly robotics conference, run by the engineering society IEEE, covering everything from robot arms and drones to self-driving cars., Physical IntelligencePhysical Intelligence, prop. n.: A San Francisco company, founded in 2024, that builds general-purpose AI 'brains' meant to control many different kinds of robots doing many different tasks., Pieter AbbeelPieter Abbeel, prop. n.: A Belgian-American roboticist at Berkeley whose research taught robots to learn from demonstrations and from trial and error, and who co-founded the robotics company Covariant., Sergey LevineSergey Levine, prop. n.: A Berkeley computer scientist, one of the most prolific researchers on how robots can learn skills from data and experience, and a co-founder of the robotics company Physical Intelligence., soft actor–criticsoft actor–critic, n.: A method for training AI agents, popular in robotics, that rewards the agent both for succeeding and for keeping its behavior varied, which helps it explore and learn reliably from reused experience.