Chat Completions¶
Write your calls once, in the OpenAI shape, and reach any model your organisation approves: GPT, Claude, Gemini or a model on Bedrock. The gateway translates the request for the provider and the answer back, so switching a model is a change of name, not a rewrite, and every call is checked, charged and recorded on the way.
What it is¶
POST /v1/chat/completions, OpenAI's chat endpoint. It's the one OpenAI endpoint the gateway translates for every
provider:
| Provider | What the gateway does |
|---|---|
| OpenAI, OpenAI-compatible providers (Mistral, Groq, Together AI, …) | Passes the request on unchanged |
| Azure OpenAI | Rewrites the address and sign-in for your Azure deployment; the body is unchanged |
| Anthropic | Translates to Anthropic's Messages format and the answer back |
| Google Gemini, Google Vertex AI | Translates to Gemini's generateContent format and the answer back |
| AWS Bedrock | Claude in Anthropic's format; Nova, Llama, Mistral and Cohere through Bedrock's Converse API (text only) |
The two settings¶
The same as for the OpenAI SDK: base address https://<your-gateway>/v1 and your Vidai key (starts vk_). See
OpenAI SDK for Python and
OpenAI SDK for TypeScript.
export OPENAI_BASE_URL="https://<your-gateway>/v1"
export OPENAI_API_KEY="vk_…"
A minimal call¶
from openai import OpenAI
client = OpenAI() # reads OPENAI_BASE_URL and OPENAI_API_KEY
reply = client.chat.completions.create(
model="claude-sonnet-4.5", # or gpt-4o-mini, gemini-3.5-flash, …
messages=[{"role": "user", "content": "Say hello in Dutch."}],
)
print(reply.choices[0].message.content)
import OpenAI from "openai";
const client = new OpenAI(); // reads OPENAI_BASE_URL and OPENAI_API_KEY
const reply = await client.chat.completions.create({
model: "claude-sonnet-4.5", // or gpt-4o-mini, gemini-3.5-flash, …
messages: [{ role: "user", content: "Say hello in Dutch." }],
});
console.log(reply.choices[0].message.content);
curl https://<your-gateway>/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "claude-sonnet-4.5", "messages": [{"role": "user", "content": "Say hello in Dutch."}]}'
The answer is in the OpenAI shape whichever provider served it.
Streaming¶
stream = client.chat.completions.create(
model="gemini-3.5-flash",
messages=[{"role": "user", "content": "Count to five."}],
stream=True,
stream_options={"include_usage": True}, # optional: a last chunk with token counts
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")
const stream = await client.chat.completions.create({
model: "gemini-3.5-flash",
messages: [{ role: "user", content: "Count to five." }],
stream: true,
stream_options: { include_usage: true }, // optional: a last chunk with token counts
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
curl -N https://<your-gateway>/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "gemini-3.5-flash", "stream": true, "messages": [{"role": "user", "content": "Count to five."}]}'
- A stream from Claude or Gemini arrives as OpenAI chunks (
data: {…}lines ending withdata: [DONE]). - The usage chunk comes only when you ask for it with
stream_options.include_usage, as with OpenAI itself. Vidai records the tokens and the cost either way. - If the provider fails before the first byte, the gateway can try a fallback model. Once the stream has started it
never switches: an error mid-stream closes the connection, and the call is recorded as
upstream_closed.
Tool calls¶
tools and tool_choice work as OpenAI documents them, on every provider: tool calls come back in
message.tool_calls, and you send results as role: "tool" messages.
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Weather for a city",
"parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
},
}]
reply = client.chat.completions.create(
model="claude-sonnet-4.5",
messages=[{"role": "user", "content": "Weather in Utrecht?"}],
tools=tools,
)
print(reply.choices[0].message.tool_calls)
const reply = await client.chat.completions.create({
model: "claude-sonnet-4.5",
messages: [{ role: "user", content: "Weather in Utrecht?" }],
tools: [{
type: "function",
function: {
name: "get_weather",
description: "Weather for a city",
parameters: { type: "object", properties: { city: { type: "string" } }, required: ["city"] },
},
}],
});
console.log(reply.choices[0].message.tool_calls);
Images in messages (image_url parts) and structured output (response_format) also reach every provider. For
images sent to another provider, a base64 data URL is the most predictable choice. Bedrock models other than Claude
take text only.
Reading what the gateway did¶
raw = client.chat.completions.with_raw_response.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hi"}],
)
print(raw.headers.get("x-vidai-request-id")) # quote this; paste it into Request logs › Request reference
print(raw.headers.get("x-vidai-model")) # the model that answered
print(raw.headers.get("x-vidai-adapted")) # what translation changed, if anything
reply = raw.parse()
const { data: reply, response } = await client.chat.completions
.create({ model: "gpt-4o-mini", messages: [{ role: "user", content: "Hi" }] })
.withResponse();
console.log(response.headers.get("x-vidai-request-id")); // paste into Request logs › Request reference
console.log(response.headers.get("x-vidai-model")); // the model that answered
console.log(response.headers.get("x-vidai-adapted")); // what translation changed, if anything
curl -sD - -o /dev/null https://<your-gateway>/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
-d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hi"}]}' | grep -i '^x-vidai'
x-vidai-adapted names any parameter the target provider can't express and that was dropped, clamped or approximated
(for example seed,top_k): the call still succeeds. Other headers you may see: x-vidai-requested-model,
x-vidai-routing-rule, x-vidai-provider-key, x-vidai-fallback, x-vidai-fallback-from,
x-vidai-fallback-attempts, x-vidai-translation-profile and x-vidai-pseudonymised. The full list is in
Headers and errors.
Errors¶
Errors come in the OpenAI format whichever provider was asked, so the SDKs raise their usual exceptions. Match on the
x-vidai-error-code header.
| Status | Common gateway codes |
|---|---|
| 400 | bad_request, provider_override_not_allowed |
| 401 | missing_api_key, invalid_api_key, conflicting_auth_headers |
| 403 | api_key_disabled, model_not_allowed, denied_by_routing_rule, content_guardrail_blocked |
| 404 | model_not_found, endpoint_not_found |
| 413 | payload_too_large |
| 429 | rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded |
| 5xx | ai_traffic_suspended, fallback_exhausted, upstream_timeout, upstream_unreachable, upstream_closed |
A provider's own error passes through with its status and message, in the OpenAI format. denied_by_routing_rule
adds rule_id, rule_name and model to the error body.
import openai
try:
client.chat.completions.create(model="gpt-4o-mini", messages=[{"role": "user", "content": "Hi"}])
except openai.APIStatusError as e:
print(e.status_code, e.response.headers.get("x-vidai-error-code"))
try {
await client.chat.completions.create({ model: "gpt-4o-mini", messages: [{ role: "user", content: "Hi" }] });
} catch (e) {
if (e instanceof OpenAI.APIError) {
console.log(e.status, e.headers?.get("x-vidai-error-code"));
}
}
Model names¶
Send the names your organisation approved, as listed under Models in the Control room (for example gpt-4o-mini,
claude-sonnet-4.5, gemini-3.5-flash). GET /v1/models (client.models.list()) is answered by the gateway and
lists exactly the models your key may call. An alias works like any other name; x-vidai-model says which model
really answered.
What goes through the gateway¶
| Feature | Through the gateway |
|---|---|
| Messages, system prompts, streaming | Yes, every provider |
Tools and tool_choice |
Yes, every provider (Bedrock non-Claude models: no tool_choice: "none") |
| Images in messages | Yes; Bedrock models other than Claude take text only |
Structured output (response_format) |
Yes; schema-less JSON mode has no Bedrock Converse equivalent |
Parameters a provider lacks (seed, top_k, …) |
Dropped or approximated and named in x-vidai-adapted |
GET /v1/models |
Answered by the gateway: the models your key may call |
Check it works in a minute¶
curl https://<your-gateway>/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
-d '{"model": "gpt-4o-mini", "max_tokens": 5, "messages": [{"role": "user", "content": "ping"}]}'
A short answer means the address and key are right. Try it again with a Claude or Gemini model name: same request, same answer shape. The call shows in Request logs within seconds.
Good to know¶
- The gateway's own 429 has no
Retry-Afterheader; back off on your side. A provider's own rate-limit headers pass through. - The OpenAI SDKs retry twice by default. A fallback chain may already have tried another model before you see an error; lower the SDK's retries if you don't want both.
- Audio input parts in a message aren't carried to another provider.