What is an AI gateway?

An AI gateway is a service between your application and AI model providers. Your code calls one API with one key; the gateway chooses the model and provider, retries elsewhere when a provider fails, applies your budgets and rules, and records what each request cost.

Start free Read the docs

POST /v1/chat/completions

Check the keyproduction-api · budget ok · IP allowed
Read the needstools · 2,100 tokens in
Shortlist8 models qualify at balanced
Rankhealthy first, then price
Sendanthropic / claude-haiku-5-5
Record$0.000290 · 1.9 s

Without and with a gateway

Calling providers directly

A key, an SDK and a bill per provider
Fallback code written by you
Spend spread across dashboards
Rules enforced in your app

Through a gateway

One key, one API, one bill
Failover handled for you
Every request costed in one place
Budgets and rules per key

An engraving of a wall-mounted telephone transmitter from a 1906 telephone handbook. — From a 1906 handbook on telephone exchanges.
The practical telephone handbook and guide to the telephonic exchange, 1906. Internet Archive Book Images, no known copyright restrictions, via Wikimedia Commons.

What a gateway usually does

Unified API: One request format for every model, usually OpenAI's Chat Completions.
Routing: Choosing the model, or the provider for a model, for each request.
Failover: Moving a request to another provider when one times out or errors.
Cost and usage tracking: Tokens, price and latency for every request.
Governance: Budgets, allowed models, guardrails, roles and audit logs.

Hosted or self-hosted

	Hosted (like DigitSpace)	Self-hosted proxy
Set-up	Sign up and change a URL	Deploy and run a service
Provider keys	The gateway's credits, or your own	Your own
Upgrades and uptime	The gateway's job	Yours
Data path	Through the gateway's servers	Through yours

When you need one

You probably don't need a gateway for one prototype on one model. You start to need one when you use more than one provider, when an outage would hurt, when several people or features share a bill, or when you have to prove where requests went.

Questions

A telephone exchange room around 1900, with a long wooden switchboard, operator stools and a desk. — General Telephone Company of the Northwest, exchange room, about 1900, public domain, cropped, via Wikimedia Commons.

Is an AI gateway the same as an API gateway?

It's a specialised one: on top of routing traffic it understands models, tokens, prices and provider health.

Does a gateway add latency?

A well-built one adds very little, since routing is decided in memory before the call. Most time is the model generating its answer.

Does it change model output?

No. The model's answer is passed back as it is; the gateway only changes which model or provider answers.

Live in a minute.

Create a key, change the base URL, send a request. No card needed with your own provider keys.

Start free