Skip to content

Responses

Keep client.responses.create(...) as it is and route it through the gateway: each call is checked against your policies, charged to the right team and recorded in Request logs, like a chat call.

What it is

POST /v1/responses, OpenAI's newer endpoint, with the calls that address a stored response by id: GET /v1/responses/{id}, POST /v1/responses/{id}/cancel and GET /v1/responses/{id}/input_items.

The gateway passes Responses calls to the provider unchanged. It reaches OpenAI and Azure OpenAI, and a provider added with the OpenAI kind of API that offers the endpoint itself. It isn't translated: a Responses call to a Claude or Gemini model gets a 400 and is never sent elsewhere. To reach those models with one request shape, use Chat Completions.

An admin can say per provider whether it serves Responses: Providers › (a provider) › Connection › Responses API: Served, Not served or The gateway's default for its kind.

The two settings

The same as for the OpenAI SDK: base address https://<your-gateway>/v1 and your Vidai key. See OpenAI SDK for Python.

A minimal call

from openai import OpenAI

client = OpenAI()  # reads OPENAI_BASE_URL and OPENAI_API_KEY

response = client.responses.create(
    model="gpt-4.1",
    input="Say hello in Dutch.",
)
print(response.output_text)
import OpenAI from "openai";

const client = new OpenAI(); // reads OPENAI_BASE_URL and OPENAI_API_KEY

const response = await client.responses.create({
  model: "gpt-4.1",
  input: "Say hello in Dutch.",
});
console.log(response.output_text);
curl https://<your-gateway>/v1/responses \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4.1", "input": "Say hello in Dutch."}'

Structured output works as OpenAI documents it for this endpoint (text={"format": {"type": "json_schema", …}}), passed on unchanged.

Streaming

stream = client.responses.create(model="gpt-4.1", input="Count to five.", stream=True)
for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="")
const stream = await client.responses.create({ model: "gpt-4.1", input: "Count to five.", stream: true });
for await (const event of stream) {
  if (event.type === "response.output_text.delta") process.stdout.write(event.delta);
}

The events are the provider's own, passed through. A fallback model is tried only before the first byte; an error after the stream started closes the connection.

Tool calls

Function tools work as OpenAI documents them; each round trip is a call of its own in Request logs. OpenAI's built-in tools (web search, file search, code interpreter, computer use) run at the provider, inside the one call: the gateway sees and records the call, not what the tool did there.

response = client.responses.create(
    model="gpt-4.1",
    input="Weather in Utrecht?",
    tools=[{
        "type": "function",
        "name": "get_weather",
        "description": "Weather for a city",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
    }],
)
print(response.output)

Stored responses and conversations

previous_response_id continues a conversation the provider stored. That works when the follow-up reaches the same provider account, which it does when you call the same model. A call that names a response by id carries no model, so the gateway finds the provider this way:

  1. the x-provider header, if you send one;
  2. otherwise the one provider whose Responses API is Served;
  3. otherwise a 400 that asks you for the x-provider header. The gateway never guesses where a stored response lives.
client.responses.retrieve("resp_…", extra_headers={"x-provider": "openai-eu"})

The provider names are the ones under Providers in the Control room.

Reading what the gateway did

raw = client.responses.with_raw_response.create(model="gpt-4.1", input="Hi")
print(raw.headers.get("x-vidai-request-id"))  # paste into Request logs › Request reference
print(raw.headers.get("x-vidai-model"))       # the model that answered
response = raw.parse()
const { data, response } = await client.responses.create({ model: "gpt-4.1", input: "Hi" }).withResponse();
console.log(response.headers.get("x-vidai-request-id"));
console.log(response.headers.get("x-vidai-model"));

Errors

The common codes are those of Chat Completions. Specific to Responses:

Status Code When
400 bad_request The model is on a provider that doesn't speak the OpenAI API (Claude, Gemini), or its provider's Responses API is Not served
400 bad_request A call by response id, with no x-provider and more than one provider that could hold it
400 provider_override_not_allowed x-provider names a provider that doesn't serve the model
404 (the provider's own) The provider doesn't know the endpoint or the stored response; the message keeps the provider's words and adds the likely cause

Model names

Use a model on OpenAI or Azure OpenAI from those approved under Models (for example gpt-4.1). GET /v1/models lists every model your key may call, including ones on providers that don't serve Responses.

What goes through the gateway

Feature Through the gateway
responses.create, streaming, function tools, structured output Yes, to OpenAI and Azure OpenAI
responses.retrieve, cancel, input_items.list Yes, to the provider found by the rule above
OpenAI's built-in tools Run at the provider; the call is recorded
Claude, Gemini and Bedrock models No: a 400. Use Chat Completions

Check it works in a minute

curl https://<your-gateway>/v1/responses \
  -H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o-mini", "input": "ping", "max_output_tokens": 16}'

Good to know

  • On Responses, a guardrail set to mask blocks the call instead.
  • A routing rule can't send a Responses call to a provider of another kind; the Control room warns before saving one.
  • The old variable OPENAI_API_BASE is from the SDK before 1.0; the Responses API needs a recent SDK, which reads OPENAI_BASE_URL.