OpenAI-compatible API
In plain wordsAn AI service that accepts the same request format as OpenAI's, so software written for OpenAI's models can use other companies' or self-hosted models by changing an address and key.
A web API — offered by a model provider, inference server or AI gatewayAI gateway, n.: A checkpoint between an organization's apps and the AI models they use that handles access, costs, logging, safety checks and switching between providers in one place. — that implements the request and response formats of OpenAIOpenAI, prop. n.: The American company that makes ChatGPT and the GPT models. It began in 2015 as a nonprofit research lab and is now a for-profit company overseen by a nonprofit foundation.'s API, chiefly the Chat Completions endpoint (/v1/chat/completions), so that OpenAI's client libraries and the many tools built on them work unchanged when pointed at a different base URL. A Chat Completions request is a list of role-tagged messages plus parameters such as model, temperaturetemperature, n.: A setting that controls how adventurous an AI model's word choices are. Low temperature makes it pick the safest, most likely words; high temperature makes it more varied and surprising, but more error-prone., tools and response format; streamed responses arrive as server-sent events. Open-source servers such as vLLMvLLM, prop. n.: A free, open-source program for running large language models on servers efficiently, so one machine can answer many more users at once. Many companies use it to host AI models., llama.cppllama.cpp, prop. n.: A free, open-source program that runs AI language models on ordinary computers and phones, without needing a powerful graphics card, by using compact low-precision versions of the models., OllamaOllama, prop. n.: A free app that makes it easy to download and run AI language models on your own computer with a single command, keeping your data on your machine. and SGLangSGLang, prop. n.: A free, open-source engine for running AI language and image models on servers, built to be fast when many requests begin the same way, as in chat and agent workloads., routers such as OpenRouter and LiteLLM, and many commercial providers offer this compatibility, making it a de facto interface standard that lowers switching costs. Compatibility is often partial: structured outputsstructured outputs, n.: Getting an AI model to answer in an exact, machine-readable format — like a form with fixed fields — so other software can use the answer without guesswork., logprobslog probability, n.: The logarithm of the probability a model gives to a token or a text. Computers use it because multiplying many tiny probabilities runs into rounding problems, whereas adding their logarithms does not., reasoning content, caching controls and newer endpoints such as OpenAI's Responses API vary between implementations, so behavior must be tested rather than assumed.
Because the server is OpenAI-compatible, we only changed the base URL to move the app onto our own GPUs.
Origin
Followed OpenAI's March 2023 launch of the Chat Completions API with the gpt-3.5-turbo model, which quickly became the most widely used language-model interface. Open-source projects began imitating it during 2023, and by 2024 most inference providers and local runtimes offered OpenAI-compatible endpoints alongside their native APIs.
Usage note
An informal convention, not a specification from a standards body; OpenAI can change its API at will. Anthropic, Google and others keep distinct native APIs while also offering compatibility layers. The Model Context ProtocolModel Context Protocol, prop. n.: An open standard that lets AI assistants plug into outside apps and data — files, calendars, databases, business software — through one common connector instead of a custom link for every pairing; often likened to a USB-C port for AI. standardizes connections to tools, not model calls.
Related
- model APImodel API, n.: A web service through which programs send requests to an AI model run by a company and receive its answers, usually paying per use instead of running the model themselves.
- AI gatewayAI gateway, n.: A checkpoint between an organization's apps and the AI models they use that handles access, costs, logging, safety checks and switching between providers in one place.
- inference providerinference provider, n.: A company that runs AI models on its own computers and sells access to them over the internet, charging by use, so customers need not buy or operate the hardware themselves.
- vLLMvLLM, prop. n.: A free, open-source program for running large language models on servers efficiently, so one machine can answer many more users at once. Many companies use it to host AI models.
- OllamaOllama, prop. n.: A free app that makes it easy to download and run AI language models on your own computer with a single command, keeping your data on your machine.
- structured outputsstructured outputs, n.: Getting an AI model to answer in an exact, machine-readable format — like a form with fixed fields — so other software can use the answer without guesswork.
Mentioned in
model APImodel API, n.: A web service through which programs send requests to an AI model run by a company and receive its answers, usually paying per use instead of running the model themselves., completioncompletion, n.: The text an AI language model writes in response to a prompt — literally, its continuation of the text it was given., inference providerinference provider, n.: A company that runs AI models on its own computers and sells access to them over the internet, charging by use, so customers need not buy or operate the hardware themselves., llama.cppllama.cpp, prop. n.: A free, open-source program that runs AI language models on ordinary computers and phones, without needing a powerful graphics card, by using compact low-precision versions of the models., OllamaOllama, prop. n.: A free app that makes it easy to download and run AI language models on your own computer with a single command, keeping your data on your machine., vLLMvLLM, prop. n.: A free, open-source program for running large language models on servers efficiently, so one machine can answer many more users at once. Many companies use it to host AI models., AI gatewayAI gateway, n.: A checkpoint between an organization's apps and the AI models they use that handles access, costs, logging, safety checks and switching between providers in one place.