Responses¶
Keep client.responses.create(...) as it is and route it through the gateway: each call is checked against your
policies, charged to the right team and recorded in Request logs, like a chat call.
What it is¶
POST /v1/responses, OpenAI's newer endpoint, with the calls that address a stored response by id:
GET /v1/responses/{id}, POST /v1/responses/{id}/cancel and GET /v1/responses/{id}/input_items.
The gateway passes Responses calls to the provider unchanged. It reaches OpenAI and Azure OpenAI, and a provider added with the OpenAI kind of API that offers the endpoint itself. It isn't translated: a Responses call to a Claude or Gemini model gets a 400 and is never sent elsewhere. To reach those models with one request shape, use Chat Completions.
An admin can say per provider whether it serves Responses: Providers › (a provider) › Connection › Responses API: Served, Not served or The gateway's default for its kind.
The two settings¶
The same as for the OpenAI SDK: base address https://<your-gateway>/v1 and your Vidai key. See
OpenAI SDK for Python.
A minimal call¶
from openai import OpenAI
client = OpenAI() # reads OPENAI_BASE_URL and OPENAI_API_KEY
response = client.responses.create(
model="gpt-4.1",
input="Say hello in Dutch.",
)
print(response.output_text)
import OpenAI from "openai";
const client = new OpenAI(); // reads OPENAI_BASE_URL and OPENAI_API_KEY
const response = await client.responses.create({
model: "gpt-4.1",
input: "Say hello in Dutch.",
});
console.log(response.output_text);
curl https://<your-gateway>/v1/responses \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "gpt-4.1", "input": "Say hello in Dutch."}'
Structured output works as OpenAI documents it for this endpoint (text={"format": {"type": "json_schema", …}}),
passed on unchanged.
Streaming¶
stream = client.responses.create(model="gpt-4.1", input="Count to five.", stream=True)
for event in stream:
if event.type == "response.output_text.delta":
print(event.delta, end="")
const stream = await client.responses.create({ model: "gpt-4.1", input: "Count to five.", stream: true });
for await (const event of stream) {
if (event.type === "response.output_text.delta") process.stdout.write(event.delta);
}
The events are the provider's own, passed through. A fallback model is tried only before the first byte; an error after the stream started closes the connection.
Tool calls¶
Function tools work as OpenAI documents them; each round trip is a call of its own in Request logs. OpenAI's built-in tools (web search, file search, code interpreter, computer use) run at the provider, inside the one call: the gateway sees and records the call, not what the tool did there.
response = client.responses.create(
model="gpt-4.1",
input="Weather in Utrecht?",
tools=[{
"type": "function",
"name": "get_weather",
"description": "Weather for a city",
"parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
}],
)
print(response.output)
Stored responses and conversations¶
previous_response_id continues a conversation the provider stored. That works when the follow-up reaches the same
provider account, which it does when you call the same model. A call that names a response by id carries no model,
so the gateway finds the provider this way:
- the
x-providerheader, if you send one; - otherwise the one provider whose Responses API is Served;
- otherwise a 400 that asks you for the
x-providerheader. The gateway never guesses where a stored response lives.
client.responses.retrieve("resp_…", extra_headers={"x-provider": "openai-eu"})
The provider names are the ones under Providers in the Control room.
Reading what the gateway did¶
raw = client.responses.with_raw_response.create(model="gpt-4.1", input="Hi")
print(raw.headers.get("x-vidai-request-id")) # paste into Request logs › Request reference
print(raw.headers.get("x-vidai-model")) # the model that answered
response = raw.parse()
const { data, response } = await client.responses.create({ model: "gpt-4.1", input: "Hi" }).withResponse();
console.log(response.headers.get("x-vidai-request-id"));
console.log(response.headers.get("x-vidai-model"));
Errors¶
The common codes are those of Chat Completions. Specific to Responses:
| Status | Code | When |
|---|---|---|
| 400 | bad_request |
The model is on a provider that doesn't speak the OpenAI API (Claude, Gemini), or its provider's Responses API is Not served |
| 400 | bad_request |
A call by response id, with no x-provider and more than one provider that could hold it |
| 400 | provider_override_not_allowed |
x-provider names a provider that doesn't serve the model |
| 404 | (the provider's own) | The provider doesn't know the endpoint or the stored response; the message keeps the provider's words and adds the likely cause |
Model names¶
Use a model on OpenAI or Azure OpenAI from those approved under Models (for example gpt-4.1).
GET /v1/models lists every model your key may call, including ones on providers that don't serve Responses.
What goes through the gateway¶
| Feature | Through the gateway |
|---|---|
responses.create, streaming, function tools, structured output |
Yes, to OpenAI and Azure OpenAI |
responses.retrieve, cancel, input_items.list |
Yes, to the provider found by the rule above |
| OpenAI's built-in tools | Run at the provider; the call is recorded |
| Claude, Gemini and Bedrock models | No: a 400. Use Chat Completions |
Check it works in a minute¶
curl https://<your-gateway>/v1/responses \
-H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
-d '{"model": "gpt-4o-mini", "input": "ping", "max_output_tokens": 16}'
Good to know¶
- On Responses, a guardrail set to mask blocks the call instead.
- A routing rule can't send a Responses call to a provider of another kind; the Control room warns before saving one.
- The old variable
OPENAI_API_BASEis from the SDK before 1.0; the Responses API needs a recent SDK, which readsOPENAI_BASE_URL.