evals-driven development
In plain wordsBuilding an AI product by first writing tests that show what good answers look like, then accepting changes to prompts, models or code only when the test scores improve.
A development methodology for AI applications, modeled on test-driven development, in which a suite of evalsevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews. defines success before or alongside the system and every change — to prompts, models, retrieval, tools or harnessharness, n.: The software around an AI model that gives it instructions and tools, carries out its actions and decides when it stops. The model is the engine; the harness is the rest of the car. — is accepted only if the suite's scores improve without regressions. The suite typically combines a curated golden datasetgolden dataset, n.: A small, carefully checked set of example questions with known good answers, used as a trusted yardstick to test an AI system every time it changes. of representative and adversarial cases; code-based checks such as format validation, exact match and unit tests on generated code; rubric scoring by model gradersLLM-as-a-judge, n.: Using one AI language model to grade or compare the answers of another, as a cheaper and faster stand-in for human reviewers. calibrated against human labels; and cases drawn from production traces through LLM observabilityLLM observability, n.: Recording and inspecting what an AI application actually did on each request — prompts sent, documents found, tools used, answers given, time and cost — so problems can be found and fixed.. Error analysis of failures drives both fixes and new test cases. Because model outputs are stochastic, scores are compared over repeated samples with attention to variance. The approach replaces informal spot checks ('vibe checks') with measurable progress and makes model upgrades and vendor switches safer.
We practice evals-driven development: no prompt change merges unless the 400-case suite holds steady or improves.
Origin
The idea circulated among practitioners in 2023–2024, promoted by OpenAI's open-source Evals framework (March 2023) and widely read essays such as Hamel Husain's 'Your AI Product Needs Evals' (2024). The phrase, also as 'eval-driven development', spread through AI-engineering talks, courses and vendor documentation in 2024–2025.
Usage note
Evals are only as good as their coverage and graders; a suite that is too small or too easy can lock in false confidence (see Goodhart's lawGoodhart's law, n.: The rule of thumb that once a number is used as a target, people or AIs find ways to push it up that no longer reflect what it was meant to measure.).
Related
- evaluationevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews.
- golden datasetgolden dataset, n.: A small, carefully checked set of example questions with known good answers, used as a trusted yardstick to test an AI system every time it changes.
- LLM-as-a-judgeLLM-as-a-judge, n.: Using one AI language model to grade or compare the answers of another, as a cheaper and faster stand-in for human reviewers.
- LLM observabilityLLM observability, n.: Recording and inspecting what an AI application actually did on each request — prompts sent, documents found, tools used, answers given, time and cost — so problems can be found and fixed.
- AI engineeringAI engineering, n.: The craft of building useful, reliable products on top of AI models — choosing models, writing prompts, connecting data and tools, testing quality and running it all safely at scale.
- prompt optimizationprompt optimization, n.: Using software, often another AI model, to search automatically for the wording and examples that make an AI model perform best on a task, instead of tweaking prompts by hand.
- vibe checkvibe check, n.: Trying out a new AI model informally — chatting with it, asking your favorite test questions — to get a gut feeling for how good it is, rather than measuring it with formal tests.
Mentioned in
evaluationevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews., AI engineeringAI engineering, n.: The craft of building useful, reliable products on top of AI models — choosing models, writing prompts, connecting data and tools, testing quality and running it all safely at scale., golden datasetgolden dataset, n.: A small, carefully checked set of example questions with known good answers, used as a trusted yardstick to test an AI system every time it changes., DSPyDSPy, prop. n.: A free software framework in which programmers describe what each step of an AI pipeline should do, and the framework automatically works out good prompts and examples instead of hand-writing them., LLM observabilityLLM observability, n.: Recording and inspecting what an AI application actually did on each request — prompts sent, documents found, tools used, answers given, time and cost — so problems can be found and fixed., LLMOpsLLMOps, n.: The everyday work of running chatbots and other products built on large language models: tracking prompts, testing answers, watching cost and quality, and updating safely when models change., model deprecationmodel deprecation, n.: When an AI company announces that an older model will stop being offered and then switches it off, so the apps and people relying on it must move to a newer one., prompt managementprompt management, n.: Treating the written instructions given to AI models like valuable code: storing them in one place, tracking every change, testing new versions and rolling back bad ones., prompt optimizationprompt optimization, n.: Using software, often another AI model, to search automatically for the wording and examples that make an AI model perform best on a task, instead of tweaking prompts by hand., vibe engineeringvibe engineering, n.: Building serious software with AI coding assistants while still taking full responsibility for its quality — the careful, professional opposite of vibe coding.