machine learning technical debt

Long-term maintenance costs specific to systems that contain machine-learning models, beyond the ordinary technical debt of software. Because a model's behavior depends on data rather than explicit code, many of the problems are properties of the whole system. The catalog compiled by D. Sculley and colleagues at Google includes entanglement, summarized as CACE ('changing anything changes everything'), where altering one input feature or setting shifts the model's behavior everywhere; hidden feedback loops in which a model influences its own future training data; undeclared consumers that silently depend on a model's outputs; unstable or underused data dependencies; configuration debt; and system anti-patterns such as glue code, 'pipeline jungles' of sprawling preprocessing and dead experimental code paths. The concept framed the case for MLOpsMLOps, n.: The working habits and tools that keep machine-learning models dependable inside real products once the experiment is over: automated testing, release, monitoring and retraining; the factory and maintenance crew around a model, not the model itself. practices such as data validation, feature storesfeature store, n.: A shared library of ready-made input signals for models — such as a customer's spending last week — computed one consistent way and supplied both when models are trained and when they run live., dependency tracking, monitoringmodel monitoring, n.: Continuously checking an AI model after it goes live — what data it receives, what it predicts and whether it is still right — so that problems are caught before they cause harm. and reproducible pipelinesmachine learning pipeline, n.: An automated chain of steps that takes raw data, prepares it, trains and tests a model and hands it over for use, so the whole process can be rerun the same way., and it is now applied to language-modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. applications, whose prompts, retrieval indexes and vendor model versions create analogous dependencies.

Every team reading this model's scores without telling us is technical debt we'll pay at the next retrain.