Remote Labor Index
In plain wordsA test that gives AI agents real paid freelance projects — design, video, data work and more — and checks how many they finish to a standard a client would accept. Early agents managed very few.
A benchmark that measures how much paid remote work AI agentsAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done. can take over from start to finish, using genuine projects commissioned from freelance professionals on online work platforms. Each project includes the original brief, the input files and the freelancer's accepted deliverable, along with what the work cost and how long it took, across fields such as product design, architecture, video and animation, game development, data analysis and writing. Trained evaluators compare an agent's output with the human deliverable and judge whether a reasonable client would accept it; the headline metric is the automation rate, the share of projects completed to that standard. When it launched in October 2025, no agent automated more than 2.5% of the projects — far lower than scores on benchmarks such as GDPvalGDPval, prop. n.: An OpenAI test that pits AI against seasoned professionals on genuine workplace assignments, from reports and spreadsheets to slide decks, in 44 occupations important to the US economy. or SWE-benchSWE-bench, prop. n.: A test that asks AI to fix real bugs and add features in real open-source Python projects, checked by running each project's own tests; the best-known yardstick for AI coding agents. might suggest — highlighting the gap between test performance and complete, client-ready work.
Despite strong benchmark scores, the agent cleared only a handful of Remote Labor Index projects.
Origin
Introduced by the Center for AI Safety and Scale AI on 30 October 2025, in a paper led by Mantas Mazeika.
Usage note
The projects are a sample of work one remote freelancer could do, and the automation rate counts only fully completed projects, not AI's partial contribution to them.
Related
- GDPvalGDPval, prop. n.: An OpenAI test that pits AI against seasoned professionals on genuine workplace assignments, from reports and spreadsheets to slide decks, in 44 occupations important to the US economy.
- task exposuretask exposure, n.: A measure of how much of a job's work AI could, in principle, do or speed up. High exposure means many of the job's tasks are within AI's reach, not that the job will vanish.
- technological unemploymenttechnological unemployment, n.: Joblessness caused by machines or software taking over work that people used to be paid for, faster than new kinds of work appear for them.
- automationautomation, n.: Using machines or software to do work that people would otherwise do, from factory robots to programs that sort email or write reports.
- AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done.
- METR time horizonMETR time horizon, n.: A way of measuring AI progress by asking how long a task — timed by how long it takes a skilled human — an AI agent can complete with even odds of success.
Sources
- Mazeika, M. et al. (2025). Remote Labor Index: Measuring AI Automation of Remote Work. arXiv:2510.26787. arxiv.org/abs/2510.26787
Mentioned in
GDPvalGDPval, prop. n.: An OpenAI test that pits AI against seasoned professionals on genuine workplace assignments, from reports and spreadsheets to slide decks, in 44 occupations important to the US economy.