Skip to content

Chat Completions

Write your calls once, in the OpenAI shape, and reach any model your organisation approves: GPT, Claude, Gemini or a model on Bedrock. The gateway translates the request for the provider and the answer back, so switching a model is a change of name, not a rewrite, and every call is checked, charged and recorded on the way.

What it is

POST /v1/chat/completions, OpenAI's chat endpoint. It's the one OpenAI endpoint the gateway translates for every provider:

Provider What the gateway does
OpenAI, OpenAI-compatible providers (Mistral, Groq, Together AI, …) Passes the request on unchanged
Azure OpenAI Rewrites the address and sign-in for your Azure deployment; the body is unchanged
Anthropic Translates to Anthropic's Messages format and the answer back
Google Gemini, Google Vertex AI Translates to Gemini's generateContent format and the answer back
AWS Bedrock Claude in Anthropic's format; Nova, Llama, Mistral and Cohere through Bedrock's Converse API (text only)

The two settings

The same as for the OpenAI SDK: base address https://<your-gateway>/v1 and your Vidai key (starts vk_). See OpenAI SDK for Python and OpenAI SDK for TypeScript.

export OPENAI_BASE_URL="https://<your-gateway>/v1"
export OPENAI_API_KEY="vk_…"

A minimal call

from openai import OpenAI

client = OpenAI()  # reads OPENAI_BASE_URL and OPENAI_API_KEY

reply = client.chat.completions.create(
    model="claude-sonnet-4.5",  # or gpt-4o-mini, gemini-3.5-flash, …
    messages=[{"role": "user", "content": "Say hello in Dutch."}],
)
print(reply.choices[0].message.content)
import OpenAI from "openai";

const client = new OpenAI(); // reads OPENAI_BASE_URL and OPENAI_API_KEY

const reply = await client.chat.completions.create({
  model: "claude-sonnet-4.5", // or gpt-4o-mini, gemini-3.5-flash, …
  messages: [{ role: "user", content: "Say hello in Dutch." }],
});
console.log(reply.choices[0].message.content);
curl https://<your-gateway>/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "claude-sonnet-4.5", "messages": [{"role": "user", "content": "Say hello in Dutch."}]}'

The answer is in the OpenAI shape whichever provider served it.

Streaming

stream = client.chat.completions.create(
    model="gemini-3.5-flash",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
    stream_options={"include_usage": True},  # optional: a last chunk with token counts
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
const stream = await client.chat.completions.create({
  model: "gemini-3.5-flash",
  messages: [{ role: "user", content: "Count to five." }],
  stream: true,
  stream_options: { include_usage: true }, // optional: a last chunk with token counts
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
curl -N https://<your-gateway>/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gemini-3.5-flash", "stream": true, "messages": [{"role": "user", "content": "Count to five."}]}'
  • A stream from Claude or Gemini arrives as OpenAI chunks (data: {…} lines ending with data: [DONE]).
  • The usage chunk comes only when you ask for it with stream_options.include_usage, as with OpenAI itself. Vidai records the tokens and the cost either way.
  • If the provider fails before the first byte, the gateway can try a fallback model. Once the stream has started it never switches: an error mid-stream closes the connection, and the call is recorded as upstream_closed.

Tool calls

tools and tool_choice work as OpenAI documents them, on every provider: tool calls come back in message.tool_calls, and you send results as role: "tool" messages.

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Weather for a city",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
    },
}]
reply = client.chat.completions.create(
    model="claude-sonnet-4.5",
    messages=[{"role": "user", "content": "Weather in Utrecht?"}],
    tools=tools,
)
print(reply.choices[0].message.tool_calls)
const reply = await client.chat.completions.create({
  model: "claude-sonnet-4.5",
  messages: [{ role: "user", content: "Weather in Utrecht?" }],
  tools: [{
    type: "function",
    function: {
      name: "get_weather",
      description: "Weather for a city",
      parameters: { type: "object", properties: { city: { type: "string" } }, required: ["city"] },
    },
  }],
});
console.log(reply.choices[0].message.tool_calls);

Images in messages (image_url parts) and structured output (response_format) also reach every provider. For images sent to another provider, a base64 data URL is the most predictable choice. Bedrock models other than Claude take text only.

Reading what the gateway did

raw = client.chat.completions.with_raw_response.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hi"}],
)
print(raw.headers.get("x-vidai-request-id"))  # quote this; paste it into Request logs › Request reference
print(raw.headers.get("x-vidai-model"))       # the model that answered
print(raw.headers.get("x-vidai-adapted"))     # what translation changed, if anything
reply = raw.parse()
const { data: reply, response } = await client.chat.completions
  .create({ model: "gpt-4o-mini", messages: [{ role: "user", content: "Hi" }] })
  .withResponse();
console.log(response.headers.get("x-vidai-request-id")); // paste into Request logs › Request reference
console.log(response.headers.get("x-vidai-model"));      // the model that answered
console.log(response.headers.get("x-vidai-adapted"));    // what translation changed, if anything
curl -sD - -o /dev/null https://<your-gateway>/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hi"}]}' | grep -i '^x-vidai'

x-vidai-adapted names any parameter the target provider can't express and that was dropped, clamped or approximated (for example seed,top_k): the call still succeeds. Other headers you may see: x-vidai-requested-model, x-vidai-routing-rule, x-vidai-provider-key, x-vidai-fallback, x-vidai-fallback-from, x-vidai-fallback-attempts, x-vidai-translation-profile and x-vidai-pseudonymised. The full list is in Headers and errors.

Errors

Errors come in the OpenAI format whichever provider was asked, so the SDKs raise their usual exceptions. Match on the x-vidai-error-code header.

Status Common gateway codes
400 bad_request, provider_override_not_allowed
401 missing_api_key, invalid_api_key, conflicting_auth_headers
403 api_key_disabled, model_not_allowed, denied_by_routing_rule, content_guardrail_blocked
404 model_not_found, endpoint_not_found
413 payload_too_large
429 rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded
5xx ai_traffic_suspended, fallback_exhausted, upstream_timeout, upstream_unreachable, upstream_closed

A provider's own error passes through with its status and message, in the OpenAI format. denied_by_routing_rule adds rule_id, rule_name and model to the error body.

import openai

try:
    client.chat.completions.create(model="gpt-4o-mini", messages=[{"role": "user", "content": "Hi"}])
except openai.APIStatusError as e:
    print(e.status_code, e.response.headers.get("x-vidai-error-code"))
try {
  await client.chat.completions.create({ model: "gpt-4o-mini", messages: [{ role: "user", content: "Hi" }] });
} catch (e) {
  if (e instanceof OpenAI.APIError) {
    console.log(e.status, e.headers?.get("x-vidai-error-code"));
  }
}

Model names

Send the names your organisation approved, as listed under Models in the Control room (for example gpt-4o-mini, claude-sonnet-4.5, gemini-3.5-flash). GET /v1/models (client.models.list()) is answered by the gateway and lists exactly the models your key may call. An alias works like any other name; x-vidai-model says which model really answered.

What goes through the gateway

Feature Through the gateway
Messages, system prompts, streaming Yes, every provider
Tools and tool_choice Yes, every provider (Bedrock non-Claude models: no tool_choice: "none")
Images in messages Yes; Bedrock models other than Claude take text only
Structured output (response_format) Yes; schema-less JSON mode has no Bedrock Converse equivalent
Parameters a provider lacks (seed, top_k, …) Dropped or approximated and named in x-vidai-adapted
GET /v1/models Answered by the gateway: the models your key may call

Check it works in a minute

curl https://<your-gateway>/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o-mini", "max_tokens": 5, "messages": [{"role": "user", "content": "ping"}]}'

A short answer means the address and key are right. Try it again with a Claude or Gemini model name: same request, same answer shape. The call shows in Request logs within seconds.

Good to know

  • The gateway's own 429 has no Retry-After header; back off on your side. A provider's own rate-limit headers pass through.
  • The OpenAI SDKs retry twice by default. A fallback chain may already have tried another model before you see an error; lower the SDK's retries if you don't want both.
  • Audio input parts in a message aren't carried to another provider.