Routes

Name the intent in your code; pin the model on the platform.

A Route is a tenant-prefixed name, like acme/support-chat, that your code sends in the model field. The Gateway resolves it to the model you have pinned server-side.

{ "model": "acme/support-chat", "messages": [...] }

Why route names instead of model ids

Model choice changes more often than application code should. With a Route, flipping a surface to a new model is an API call on the platform, not a pull request in your repo. Your code names the intent; the Route decides the model.

Fallbacks

A Route may declare an ordered fallback list. If the primary model cannot produce output because its provider is down or erroring, the Gateway tries your declared fallbacks in turn, so a single degraded model never fails the call.

This is not a silent swap:

• The served model is returned in the x-allocate-model response header.
• Fallbacks are billed at the served model's own price.
• A fallback only ever fires along the chain you declared. The platform never substitutes a model you did not choose.

Share and shadow

A Route may serve only a share of its traffic on the primary model, sending the rest to the fallbacks. That is how a new model earns its traffic. On a small sampled fraction of the turns that went to a fallback, the Gateway also asks the primary model the same question after your reply has been sent, compares the two answers with a verifier, and keeps the pair in RecursiveDB. The second answer never reaches a user, never enters a stream, and is never billed. Shadow calls are capped per day, so the bill stays bounded.

Policy always wins

If your organization has a provider policy, it is enforced after route resolution: a request can never reach a provider outside your allowlist, whatever the route or the model field says. See Private Inference Cloud.