AI engineering

Foundation-model applicationsemerging

The discipline of turning pretrained foundation modelsfoundation model, n.: A very large AI model, built once at great cost from vast amounts of text, images or code gathered mostly from the internet, that many products then reuse as their starting point, from chatbots to coding tools., reached through a model APImodel API, n.: A web service through which programs send requests to an AI model run by a company and receive its answers, usually paying per use instead of running the model themselves. or served from open weightsopen-weight model, n.: An AI model whose trained internal numbers are published for anyone to download and run on their own computers, even if the data and code used to make it stay private., into working applications, as opposed to training models from scratch. Its core activities are model selection; prompt engineeringprompt engineering, n.: The skill of writing requests to an AI — picking the words, examples, format and background information it receives — so that it reliably gives the result you want, without retraining it. and context engineeringcontext engineering, n.: Managing an AI model's reading material: choosing, trimming and arranging the instructions, documents, tool results and memories it is given on every turn, so that it has what the task needs and little else.; retrievalretrieval-augmented generation, n.: Giving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from. and data integration; tool usetool use, n.: An AI model's ability to reach beyond its own knowledge by operating outside tools — searching the web, running code, using a calculator or an app — and folding the results into its work., agent and harnessharness, n.: The software around an AI model that gives it instructions and tools, carries out its actions and decides when it stops. The model is the engine; the harness is the rest of the car. design; structured outputsstructured outputs, n.: Getting an AI model to answer in an exact, machine-readable format — like a form with fixed fields — so other software can use the answer without guesswork.; evaluationevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews. with golden setsgolden dataset, n.: A small, carefully checked set of example questions with known good answers, used as a trusted yardstick to test an AI system every time it changes. and model graders (evals-driven developmentevals-driven development, n.: Building an AI product by first writing tests that show what good answers look like, then accepting changes to prompts, models or code only when the test scores improve.); fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way. where prompting falls short; and operating concerns such as latencylatency, n.: How long you wait between asking an AI system for something and getting the answer. Lower latency means a snappier, more responsive system., cost, guardrailsguardrails, n.: Safety checks built around an AI system — filters on what goes in and comes out, and limits on what it may do — to keep it from producing harmful or off-topic content or leaking data., observabilityLLM observability, n.: Recording and inspecting what an AI application actually did on each request — prompts sent, documents found, tools used, answers given, time and cost — so problems can be found and fixed. and safe rollout. Compared with classical machine-learning engineering, the work shifts from feature engineering and training toward adapting general models through their context, and from measuring accuracy on fixed test sets toward evaluating open-ended outputs.

Most of our AI engineering effort goes into evals and retrieval, not into choosing the model.

Engineering discipline

More broadly, the discipline of carrying systems- and software-engineering rigor through the whole life of an AI-enabled system of any kind, from requirements and testing to assurance and maintenance, with stress on robustness, security and human-centered design. In this sense, used especially in government and defense research programs, the term encompasses MLOpsMLOps, n.: The working habits and tools that keep machine-learning models dependable inside real products once the experiment is over: automated testing, release, monitoring and retraining; the factory and maintenance crew around a model, not the model itself..