LLMOps

The adaptation of MLOpsMLOps, n.: The working habits and tools that keep machine-learning models dependable inside real products once the experiment is over: automated testing, release, monitoring and retraining; the factory and maintenance crew around a model, not the model itself. to applications built on large language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code., which are usually accessed through a model APImodel API, n.: A web service through which programs send requests to an AI model run by a company and receive its answers, usually paying per use instead of running the model themselves. or served from open weightsopen-weight model, n.: An AI model whose trained internal numbers are published for anyone to download and run on their own computers, even if the data and code used to make it stay private. rather than trained in-house. The managed artifacts shift from datasets and weights toward prompts, retrievalretrieval-augmented generation, n.: Giving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from. indexes, tooltool, n.: Any outside capability an AI model is allowed to use — a web search, a calculator, a code runner, a database or an app — described to the model so it can ask for it by name. definitions and model versions, so LLMOps emphasizes prompt managementprompt management, n.: Treating the written instructions given to AI models like valuable code: storing them in one place, tracking every change, testing new versions and rolling back bad ones., evaluationevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews. suites and golden setsgolden dataset, n.: A small, carefully checked set of example questions with known good answers, used as a trusted yardstick to test an AI system every time it changes., tracingLLM observability, n.: Recording and inspecting what an AI application actually did on each request — prompts sent, documents found, tools used, answers given, time and cost — so problems can be found and fixed. of multi-step calls, per-token cost and latencylatency, n.: How long you wait between asking an AI system for something and getting the answer. Lower latency means a snappier, more responsive system. tracking, guardrailsguardrails, n.: Safety checks built around an AI system — filters on what goes in and comes out, and limits on what it may do — to keep it from producing harmful or off-topic content or leaking data., gatewaysAI gateway, n.: A checkpoint between an organization's apps and the AI models they use that handles access, costs, logging, safety checks and switching between providers in one place. that route among providers, and fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way. pipelines where used. Because outputs are open-ended and providers update or retire models, quality must be checked continuously with automated graders and human review rather than a single accuracy metric. Variants such as GenAIOps and AgentOps extend the label to generative media and to agentsAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done..

Once the chatbot hit production, LLMOps meant versioning every prompt and replaying the eval suite before each model upgrade.