canary deployment
In plain wordsReleasing a new model or software version to a small slice of users first and watching closely, so that any problem hits only a few people before the change reaches everyone.
A release strategy in which a new version — of a model, prompt or serving software — receives a small fraction of live traffic (often 1–5 percent, or one region or cluster) while the rest stays on the current version, and its error rates, latencylatency, n.: How long you wait between asking an AI system for something and getting the answer. Lower latency means a snappier, more responsive system., business metrics and quality signals are compared with the baseline. If the canary stays healthy, traffic is raised in steps to full rollout; if it breaches thresholds, traffic is shifted back automatically or by an operator. Unlike a shadow deploymentshadow deployment, n.: Quietly running a new AI model alongside the old one on real requests, recording its answers without showing them to anyone, to check it behaves well before it takes over., canary users do see the new outputs, so real-world effects are observed, but at limited exposure. Unlike an A/B testA/B testing, n.: Comparing two versions of something — two models, two prompts, two designs — by randomly showing each to different groups of real users and measuring which works better., the aim is to catch regressions and failures quickly rather than to estimate an effect size precisely, though the two are often combined. For language-modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. applications, canaries are a common safeguard when switching model versions or providers.
The canary at two percent of traffic showed doubled timeouts, so the rollout halted automatically.
Origin
Named after the canaries once carried into coal mines to warn of toxic gas. The first use in software has not been traced; by 2010 Jez Humble and David Farley described 'canary releasing' in their book Continuous Delivery, and the practice later became standard in MLOpsMLOps, n.: The working habits and tools that keep machine-learning models dependable inside real products once the experiment is over: automated testing, release, monitoring and retraining; the factory and maintenance crew around a model, not the model itself..
Related
- shadow deploymentshadow deployment, n.: Quietly running a new AI model alongside the old one on real requests, recording its answers without showing them to anyone, to check it behaves well before it takes over.
- A/B testingA/B testing, n.: Comparing two versions of something — two models, two prompts, two designs — by randomly showing each to different groups of real users and measuring which works better.
- model deploymentmodel deployment, n.: Putting a trained AI model to work in a real product so that people or other software can use it — and doing so carefully enough that the change can be undone if something goes wrong.
- model monitoringmodel monitoring, n.: Continuously checking an AI model after it goes live — what data it receives, what it predicts and whether it is still right — so that problems are caught before they cause harm.
- service-level objectiveservice-level objective, n.: A specific, measurable target for how well a service should perform — for example, 99.9 percent of requests answered within two seconds — used to decide when engineers must act.
- shadow deploymentshadow deployment, n.: Quietly running a new AI model alongside the old one on real requests, recording its answers without showing them to anyone, to check it behaves well before it takes over.
Mentioned in
A/B testingA/B testing, n.: Comparing two versions of something — two models, two prompts, two designs — by randomly showing each to different groups of real users and measuring which works better., model deploymentmodel deployment, n.: Putting a trained AI model to work in a real product so that people or other software can use it — and doing so carefully enough that the change can be undone if something goes wrong., shadow deploymentshadow deployment, n.: Quietly running a new AI model alongside the old one on real requests, recording its answers without showing them to anyone, to check it behaves well before it takes over.