AllocateWhat is an LLM gateway?
An LLM gateway is one endpoint in front of every model: your code calls a single OpenAI-compatible API, and the gateway handles authentication, routing, metering, failover, and policy across providers. It turns model choice from a code dependency into configuration.
The gateway is where production concerns live: per-key spend caps, per-route metering, provider failover, and audit. Application code stays a stable API call while models, providers, and prices change behind it.
Overhead is the design constraint that separates gateways: the hop must cost milliseconds, not another model’s worth of latency. Allocate’s gateway holds itself to a 30 ms median overhead budget, reported on every response.
Browse the model catalogRelated terms
Allocate is the cloud inference platform for companies that want to train and run their own models.