Skip to content
Last Dictionary
Plane
Agents & HarnessesAI Engineering & MLOps

orchestration

nounAlso: agent orchestration, LLM orchestration, orchestrate, orchestrator

In plain wordsCoordinating several AI calls, tools or agents so each does its part in the right order and their results are combined into one finished piece of work, as a conductor directs an orchestra.

  1. 1

    Agents

    The coordination of multiple model calls, toolstool, n.: Any outside capability an AI model is allowed to use — a web search, a calculator, a code runner, a database or an app — described to the model so it can ask for it by name. or agentsAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done. toward one task: deciding what work to do, in what order or in parallel, routing each piece to the right component, passing state between steps, handling failures and retries, and merging the results. Orchestration may be fixed in code, as in workflowsagentic workflow, n.: A multi-step process in which AI models draft, check, use tools and revise their work over several rounds, instead of answering in one shot; often the steps are laid out by developers in advance. and prompt chainingprompt chaining, n.: A way of getting a big job done with a relay of AI requests: each handles one stage and passes its result to the next — pull out the facts, write, polish — with automatic checks between stages., or performed at run time by a model acting as orchestrator, as in the orchestrator–worker patternorchestrator–worker pattern, n.: A way of organizing AI teamwork, like a newsroom: an editor AI decides which stories a big assignment needs, sends reporter AIs to cover them at the same time, and turns what they file into one report. and other multi-agent systemsmulti-agent system, n.: A setup in which several AI agents, each with its own role or point of view, work together — or compete — on a problem, passing messages and results among themselves.. The component that does it is the orchestrator; libraries for it (agent frameworksagent framework, n.: A software library that gives developers ready-made parts for building AI agents — model connections, tool handling, memory, multi-agent coordination and logging — so they need not write everything from scratch.) are often marketed as orchestration frameworks. Good orchestration limits duplicated work and unnecessary context passing, since each extra agent multiplies token cost.

    The orchestration layer fans the query out to four research agents and waits for all of them.

  2. 2

    Computing (earlier sense)

    Automated coordination of software services or infrastructure — for example container orchestration with Kubernetes or workflow orchestration with Apache Airflow — from which the agent sense was borrowed; machine-learning pipelines are still orchestrated in this sense.

Origin

Borrowed from service-oriented computing (service orchestration, 2000s) and container orchestration (2010s). 'LLM orchestration' described frameworks such as LangChain and LlamaIndex from 2023; multi-agent orchestration became standard vocabulary with OpenAI's experimental Swarm library (October 2024) and Agents SDK (March 2025) and Anthropic's engineering posts on orchestrator–worker systems (2024–2025).

Related

See also
  • orchestrator–worker patternorchestrator–worker pattern, n.: A way of organizing AI teamwork, like a newsroom: an editor AI decides which stories a big assignment needs, sends reporter AIs to cover them at the same time, and turns what they file into one report.
  • multi-agent systemmulti-agent system, n.: A setup in which several AI agents, each with its own role or point of view, work together — or compete — on a problem, passing messages and results among themselves.
  • subagentsubagent, n.: A helper AI agent that a main agent starts to handle one piece of a job; it works in its own separate workspace and reports back a short summary when it is done.
  • agentic workflowagentic workflow, n.: A multi-step process in which AI models draft, check, use tools and revise their work over several rounds, instead of answering in one shot; often the steps are laid out by developers in advance.
  • agent frameworkagent framework, n.: A software library that gives developers ready-made parts for building AI agents — model connections, tool handling, memory, multi-agent coordination and logging — so they need not write everything from scratch.
  • handoffhandoff, n.: The moment one AI agent passes a conversation or task to another, better-suited agent — like a call-center operator transferring you to the right department.
  • harnessharness, n.: The software around an AI model that gives it instructions and tools, carries out its actions and decides when it stops. The model is the engine; the harness is the rest of the car.
Contrast with
  • agent loopagent loop, n.: The repeating cycle at the heart of an AI agent: the model decides what to do next, software carries it out, the result is shown to the model, and the cycle repeats until the task is finished.

Sources

  1. Schluntz, E. & Zhang, B. (2024). Building effective agents. Anthropic. anthropic.com/research/building-effective-agents
  2. OpenAI (2025). New tools for building agents. 11 March 2025. openai.com/index/new-tools-for-building-agents

Mentioned in

agent frameworkagent framework, n.: A software library that gives developers ready-made parts for building AI agents — model connections, tool handling, memory, multi-agent coordination and logging — so they need not write everything from scratch., agentic workflowagentic workflow, n.: A multi-step process in which AI models draft, check, use tools and revise their work over several rounds, instead of answering in one shot; often the steps are laid out by developers in advance., AI stackAI stack, n.: The layers of technology an AI product depends on, from chips and data centers at the bottom, through the models, up to the apps people use, each built on the layer below., orchestrator–worker patternorchestrator–worker pattern, n.: A way of organizing AI teamwork, like a newsroom: an editor AI decides which stories a big assignment needs, sends reporter AIs to cover them at the same time, and turns what they file into one report., prompt chainingprompt chaining, n.: A way of getting a big job done with a relay of AI requests: each handles one stage and passes its result to the next — pull out the facts, write, polish — with automatic checks between stages., AutoGenAutoGen, prop. n.: A free Microsoft software framework for building teams of AI agents that solve problems by messaging one another, writing and running code, and asking people for help when needed., compound AI systemcompound AI system, n.: An AI product built from several working parts — models, search tools, code and checks — designed to work together, rather than a single model answering on its own., CrewAICrewAI, prop. n.: A free software framework for setting up teams of AI agents, each given a job title, goal and backstory, that divide up tasks and work through them together like a small staff., handoffhandoff, n.: The moment one AI agent passes a conversation or task to another, better-suited agent — like a call-center operator transferring you to the right department., LangGraphLangGraph, prop. n.: A free software library for building AI agents as flowcharts of steps that share a memory, so developers control exactly how an agent moves between steps, pauses and resumes., LlamaIndexLlamaIndex, prop. n.: A free software toolkit, and the company behind it, for connecting AI language models to an organization's own documents and data so the models can find and use that information.