Skip to content

OpenAI SDK for Python

Your Python code keeps calling client.chat.completions.create(...). Change two settings and every call is governed: it's checked against your policies, charged to the right team and recorded in Request logs, and it can reach Claude or Gemini as well as OpenAI's models, answered in the OpenAI shape your code already reads.

What it is

The official openai package for Python (version 1.0 or later). It speaks the OpenAI request format, which the gateway accepts at /v1.

The two settings

Setting Environment variable Constructor argument Value
Base address OPENAI_BASE_URL base_url https://<your-gateway>/v1
Key OPENAI_API_KEY api_key Your Vidai key (starts vk_), never a provider's key

The gateway's address, and the base URL for each kind of SDK, are under Connect your app on the Keys page and in System › Settings › Gateway. You get the key from Keys › Issue a key; its secret is shown once.

export OPENAI_BASE_URL="https://<your-gateway>/v1"
export OPENAI_API_KEY="vk_…"

A minimal call

from openai import OpenAI

client = OpenAI()  # reads OPENAI_BASE_URL and OPENAI_API_KEY
# or: OpenAI(base_url="https://<your-gateway>/v1", api_key="vk_…")

reply = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Say hello in Dutch."}],
)
print(reply.choices[0].message.content)

The same code calls Claude or Gemini: put a model name such as claude-sonnet-4.5 in model. The gateway translates the request for that provider and the answer back into the OpenAI shape, streams and tool calls included.

Streaming

stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
    stream_options={"include_usage": True},  # optional: a last chunk with token counts
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

A stream from Claude or Gemini arrives as OpenAI chunks. The usage chunk comes only when you ask for it, as with OpenAI itself; Vidai records the tokens and the cost either way.

Tool calls

tools and tool_choice work as documented by OpenAI, on every provider the gateway translates to: the model's tool calls come back in message.tool_calls, and you send results as role: "tool" messages.

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Weather for a city",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
    },
}]
reply = client.chat.completions.create(
    model="claude-sonnet-4.5",
    messages=[{"role": "user", "content": "Weather in Utrecht?"}],
    tools=tools,
)
print(reply.choices[0].message.tool_calls)

Reading what the gateway did

Prefix the call with with_raw_response to read the headers the gateway adds, then parse() for the usual object:

raw = client.chat.completions.with_raw_response.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hi"}],
)
print(raw.headers.get("x-vidai-request-id"))  # quote this; paste it into Request logs › Request reference
print(raw.headers.get("x-vidai-model"))       # the model that answered
print(raw.headers.get("x-vidai-adapted"))     # what translation changed, if anything
reply = raw.parse()

Other headers you may see: x-vidai-requested-model (when a different model answered), x-vidai-routing-rule, x-vidai-provider-key (the provider key's label, never its secret), x-vidai-fallback and x-vidai-fallback-from, x-vidai-fallback-attempts, x-vidai-translation-profile and x-vidai-pseudonymised. Cost is never in the answer: find the call in Request logs.

Errors

The gateway writes its errors in the OpenAI format, so the SDK raises its usual exceptions. Match on the x-vidai-error-code header rather than the message: codes are stable.

Status Exception Common gateway codes
400 BadRequestError bad_request, provider_override_not_allowed, an endpoint this provider doesn't serve
401 AuthenticationError missing_api_key, invalid_api_key, conflicting_auth_headers
403 PermissionDeniedError api_key_disabled, model_not_allowed, denied_by_routing_rule, content_guardrail_blocked
404 NotFoundError model_not_found, endpoint_not_found
413 APIStatusError payload_too_large
429 RateLimitError rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded
5xx InternalServerError ai_traffic_suspended, fallback_exhausted, upstream_timeout, upstream_unreachable
import openai

try:
    client.chat.completions.create(model="gpt-4o-mini", messages=[{"role": "user", "content": "Hi"}])
except openai.APIStatusError as e:
    print(e.status_code, e.response.headers.get("x-vidai-error-code"))
    print(e.response.headers.get("x-vidai-request-id"))

A key that's switched off carries a short reason beside api_key_disabled in the error body (for example agent_suspended); treat the list of reasons as open-ended.

Model names

Send the names your organisation has approved, as listed under Models in the Control room. client.models.list() returns exactly the models your key may call. An alias set by an admin works like any other name; x-vidai-model says which model really answered.

What goes through the gateway

Feature Through the gateway
Chat Completions, streaming, tools, structured output, vision Yes, to OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models
client.models.list() Yes: answered by the gateway, only the models your key may call
Responses, embeddings, moderations, images, audio, files, batches, fine-tuning Yes, to OpenAI and Azure OpenAI only; another provider gets a clear 400
Assistants, Realtime No: see Assistants API and Realtime API

Check it works in a minute

python -c "from openai import OpenAI; print(OpenAI().chat.completions.create(model='gpt-4o-mini', max_tokens=5, messages=[{'role':'user','content':'ping'}]).choices[0].message.content)"

A short answer means the address and key are right. The call shows in Request logs within seconds.

Good to know

  • The SDK retries twice by default (on 408, 409, 429 and 5xx). A fallback chain may already have tried another model before you see an error; set max_retries=0 if you don't want both, or keep a small number for network errors.
  • The gateway's own 429 has no Retry-After header; back off on your side. A provider's own rate-limit headers pass through.
  • Anything other than chat sent to a model of another provider is refused with a 400, never sent somewhere else.
  • OPENAI_API_BASE is the variable of the SDK before 1.0; version 1.0 and later read OPENAI_BASE_URL.