dexterous manipulation

Manipulationmanipulation, n.: A robot's use of arms, hands or grippers to change the world around it: picking things up, moving, pushing, opening, assembling or using tools. that controls an object's position and orientation through coordinated, often changing contacts, typically with multi-fingered hands, rather than by clamping it rigidly: in-hand reorientation, finger gaiting, rolling or sliding an object between fingers, and using tools such as scissors or keys. It is among the hardest problems in roboticsrobotics, n.: The science and engineering of building robots and making them work: designing their bodies, sensors and motors, and writing the software that lets them see, plan and move safely.. A human hand has more than 20 degrees of freedomdegrees of freedom, n.: The number of independent ways something can move. A robot arm with six motorized joints has six; a free-floating object can slide three ways and turn three ways.; intermittent contacts, friction and occlusion make planning and state estimation difficult; and hardware is fragile and costly. Model-based approaches plan contact sequences with trajectory optimizationtrajectory optimization, n.: Calculating the best way for a robot to move over time, choosing exact positions, speeds and forces moment by moment so the motion is smooth, fast or energy-efficient and avoids obstacles.. Learning approaches train policies with reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time. in simulation and transfer them using domain randomizationdomain randomization, n.: Training a robot's AI in a simulation whose colors, lighting, weights, friction and other details are shuffled at random on every run, so that the real world holds few surprises it has not already practiced for. — as in OpenAI's Dactyl, which reoriented a block (2018) and solved a Rubik's Cube one-handed (2019) — or learn from human demonstrations captured by teleoperationteleoperation, n.: Controlling a robot or machine from a distance, with a person steering it in real time using a joystick, a matching 'puppet' arm, a VR headset or a screen. and data gloves. Tactile sensingtactile sensing, n.: Giving robots a sense of touch, with sensors in fingertips or skin that feel pressure, texture, vibration and slipping, so they can handle objects gently and securely. is widely seen as essential. By 2025–2026 vision-language-action modelsvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements. were also being used to drive multi-fingered hands on humanoidshumanoid robot, n.: A robot built roughly in human shape, with a head, torso, two arms and usually two legs, so that it can work in spaces and with tools designed for people., though reliability on harder tasks remained far below human levels.

Dexterous manipulation is where our sim-to-real pipeline still breaks: the fingers slip on real plastic.