Build a fallback chain¶
Purpose: when a provider goes down, apps without a plan return errors to their users. A fallback chain names the models to try instead, in your order. When the gateway sees a provider failing (its circuit is open), it sends calls to the next model in the chain; set it to, and it also resends a call whose model fails before answering.
In the demo world, the Creative team's calls ask for gpt-4.1-mini. Its chain answers from gpt-4.1-mini-eu, then
mistral-small-paris, when that provider is down; the steps below build it.
Before you start¶
- You're an organisation admin, or a group admin for your own teams.
- The models you'll fall back to are ones the keys at that level may call. Pick models in the same region if a region rule applies.
Steps¶
- Open Models & routing › Routing and choose New routing… › Fall back when a provider is down › Continue (or New fallback chain on the Fallback chains tab).
- Applies to: the level and which one, for example Team › Creative.
- Calls to: the provider, then the model to protect.
- Under Add a model to try, choose a provider, then one of its models, and Add to the chain. Each entry shows its provider, where it runs and its list price against the protected model. Add as many as you need.
- Put them in order with Up and Down (they work from the keyboard), or Remove one.
- When every model in the chain fails:
- Try the model asked for anyway (the default): the call goes to the original model; callers get whatever its provider answers.
- Refuse the call: callers get a "service unavailable" answer saying every fallback failed, so their program can wait and try again.
- When the model called fails (it can't be reached, times out, says too many requests or fails, before anything reached the caller):
- Return its error (the default): callers get that error, as without a chain.
- Try the next model: the call is sent again to the next model in the chain, at most 3 attempts in all.
- Check what it changes, then Save.
Check it worked¶
The chain's card on Fallback chains draws it as a flow, Asked for → Fallback 1 → Fallback 2, with what happens if every one fails and "If a call fails: try the next model." when it does.

When a chain is used, the call in Request logs says Fallback under Answered by ("answered by … after …
failed") and lists Models tried, in order, each Answered or Failed with why. A call none answered reads
Every fallback failed. Apps see it too: the answer carries x-vidai-fallback: true and x-vidai-fallback-from, and
x-vidai-fallback-attempts when more than one attempt was made.

Related¶
- Read provider health and circuits
- Keep calls in a region: a chain never sends a call outside a region rule.
- See which routing rule wins
Good to know¶
- Trying the next model happens only before anything reaches the caller, never in the middle of a streamed answer.
- A provider may charge for an attempt that failed after it received the call.
- The gateway resends at most 3 attempts, and never resends uploads, files or batches, or a call too large or too slow to send again; the call then says why it stopped trying.
- A chain acts on outages and failed calls, not on the content of a model's answer.
- A chain says Already in place when a chain above for the same model has the same entries.