Migrate from calling providers directly¶
Purpose: move an existing app onto the gateway with two changed settings, then remove what it no longer needs: provider keys in your secrets, your own failover between providers, your own cost logging. Your code, prompts and SDK stay as they are.
Before you start¶
- A key for the app, ideally an agent's key in the right environment (Keys and API tokens).
- The gateway's address (Find the gateway's address).
- The models the app uses, available to the key: check with
GET /v1/models. If a name differs from the provider's, see Model names and aliases.
Change two settings¶
client = OpenAI(
- api_key=os.environ["OPENAI_API_KEY"],
+ base_url=os.environ["VIDAI_GATEWAY"] + "/v1",
+ api_key=os.environ["VIDAI_KEY"],
)
client = Anthropic(
- api_key=os.environ["ANTHROPIC_API_KEY"],
+ base_url=os.environ["VIDAI_GATEWAY"],
+ api_key=os.environ["VIDAI_KEY"],
)
client = genai.Client(
- api_key=os.environ["GEMINI_API_KEY"],
+ api_key=os.environ["VIDAI_KEY"],
+ http_options=types.HttpOptions(base_url=os.environ["VIDAI_GATEWAY"]),
)
Many SDKs and tools read their address and key from the environment, so no code changes at all:
# OpenAI SDKs
export OPENAI_BASE_URL="https://<your-gateway>/v1"
export OPENAI_API_KEY="$VIDAI_KEY"
# Anthropic SDKs
export ANTHROPIC_BASE_URL="https://<your-gateway>"
export ANTHROPIC_API_KEY="$VIDAI_KEY"
What you can delete¶
| You had | Now | Remove |
|---|---|---|
| A provider key per app, in each app's secrets | One Vidai key per app; the providers' keys stay with the gateway | The provider keys from the app's secrets, then rotate them at the provider |
| A client per provider, to reach models from different vendors | One SDK reaches every model your key may call (Any SDK to any provider) | The extra clients, if you like |
| Your own failover from one provider to another | Fallback chains your organisation sets up, applied before you see a failure | Your cross-provider failover code |
| Your own token and cost logging | Every call priced in Request logs and Money › Cost | The cost bookkeeping |
| Your own filters for personal data in prompts | Guardrails, applied to every call | Duplicated filters, once your organisation's Guardrails cover them |
| A model name hard-coded per provider | A name your organisation can move (Model names and aliases) | Nothing: keep the name, drop the per-provider switch |
Retries¶
The gateway can try the next model in a fallback chain when a provider fails: on a connection error, a time-out, 429 or
5xx, before anything has reached you, at most 3 attempts in all. If the chain runs out, you get the last model's own
error. x-vidai-fallback-attempts on the answer says how many attempts were made. It does this only where an admin
set a chain to Try the next model (Fallback and retries).
So keep your SDK's own retries few and spaced out (the SDKs default to two, with backoff), and never retry a 401 or 403. Two layers of eager retries multiply: three of yours times three of the gateway's is nine calls to a struggling provider, and a provider may charge for an attempt that failed after it received the call.
Check it worked¶
- Run the app's usual path once.
- Open Daily › Request logs and filter by the app's Key: every call is there, with its model, outcome and cost.
- Check
x-vidai-adaptedon a few answers if a model now sits behind another provider: it names anything that couldn't be carried exactly.

Good to know¶
x-vidai-*headers are extra: a client that ignores them works the same.- The gateway doesn't send rate-limit headers of its own; a provider's pass through and describe the provider's limits, not your key's (Limits).
- Files, batches and uploads live in the provider's account the gateway uses, shared by every key on that provider (Endpoints).
- The Responses API, embeddings, images and audio work with a provider of the same format only.