domain randomization
In plain wordsTraining a robot's AI in a simulation whose colors, lighting, weights, friction and other details are shuffled at random on every run, so that the real world holds few surprises it has not already practiced for.
A sim-to-real transfersim-to-real transfer, n.: Training a robot's AI in a computer simulation, where practice is fast, cheap and safe, and then making it work on the real robot despite the differences between simulation and reality. technique that trains a model across a wide distribution of randomized simulations instead of one carefully calibrated simulation, so that it becomes robust to whatever the real world turns out to be. Visual randomization varies textures, colors, lighting, camera pose, distractor objects and noise, often deliberately unrealistically, for perception; dynamics randomization varies masses, friction, motor strength, damping, latency and sensor noise for control. The reasoning is that if the training distribution is wide enough to take in the real world's conditions, reality will not look special to the policy, and a policy with memory can implicitly identify and adapt to the conditions it finds itself in. Too much randomization makes the task needlessly hard and the policy overly cautious, so ranges are tuned by hand, inferred from real data, or widened automatically as the policy improves (automatic domain randomization, used by OpenAI to solve a Rubik's Cube with a robot hand in 2019). It is a form of data augmentationdata augmentation, n.: Creating extra training examples by changing existing ones in ways that keep their meaning — a flipped, cropped or recolored photo of a cat still shows a cat — so the model learns more robustly. and is standard in reinforcement-learned locomotionlocomotion, n.: How a robot moves itself from place to place: rolling on wheels, walking on legs, crawling, swimming or flying. and in training perception on synthetic datasynthetic data, n.: Data created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars..
We randomize floor friction between 0.3 and 1.2, and now the policy walks on wet tiles.
Origin
Named in 'Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World' by Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba and Pieter Abbeel (OpenAI and UC Berkeley, March 2017; IROS 2017), whose object detector, trained only on randomly textured renderings, located real objects to within 1.5 centimeters. Fereshteh Sadeghi and Sergey Levine's CAD2RL (2016) applied the same idea to drone flight.
Related
- sim-to-real transfersim-to-real transfer, n.: Training a robot's AI in a computer simulation, where practice is fast, cheap and safe, and then making it work on the real robot despite the differences between simulation and reality.
- physics simulatorphysics simulator, n.: Software that imitates how real objects move, collide, bend and fall, so robots and their AI can be designed, trained and tested on a computer before touching the real world.
- data augmentationdata augmentation, n.: Creating extra training examples by changing existing ones in ways that keep their meaning — a flipped, cropped or recolored photo of a cat still shows a cat — so the model learns more robustly.
- synthetic datasynthetic data, n.: Data created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars.
- reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.
- robustnessrobustness, n.: An AI system's ability to keep working correctly and safely when conditions change — unfamiliar data, noisy inputs, or people deliberately trying to trick it.
Sources
- Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W. & Abbeel, P. (2017). Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World. arXiv:1703.06907. arxiv.org/abs/1703.06907
- OpenAI et al. (2019). Solving Rubik's Cube with a Robot Hand. arXiv preprint.
Mentioned in
synthetic datasynthetic data, n.: Data created by computer programs or AI models rather than collected from the real world — such as practice math problems written by a model, or simulated street scenes for self-driving cars., domain adaptationdomain adaptation, n.: Adjusting a model trained on one kind of data, such as daytime photos or one hospital's records, so that it still works on a different but related kind, such as night photos or another hospital's., locomotionlocomotion, n.: How a robot moves itself from place to place: rolling on wheels, walking on legs, crawling, swimming or flying., physics simulatorphysics simulator, n.: Software that imitates how real objects move, collide, bend and fall, so robots and their AI can be designed, trained and tested on a computer before touching the real world., sim-to-real transfersim-to-real transfer, n.: Training a robot's AI in a computer simulation, where practice is fast, cheap and safe, and then making it work on the real robot despite the differences between simulation and reality., dexterous manipulationdexterous manipulation, n.: Skilled handling of objects with robot fingers, such as spinning a pen, turning a key or rotating a cube in one hand: the fine finger work people do without thinking., Pieter AbbeelPieter Abbeel, prop. n.: A Belgian-American roboticist at Berkeley whose research taught robots to learn from demonstrations and from trial and error, and who co-founded the robotics company Covariant.