Generate content¶
generateContent and streamGenerateContent are what Google's GenAI SDKs, the Agent Development Kit and most
Gemini-format tools call. Through the gateway they reach every provider your organisation has set up: Gemini as sent,
and OpenAI, Anthropic and the rest translated both ways, so your code reads a Gemini answer whichever model wrote it.
What it is¶
The Gemini API's content generation: POST /v1beta/models/{model}:generateContent, and
POST /v1beta/models/{model}:streamGenerateContent?alt=sse for a stream. The body has contents, and optionally
systemInstruction, tools, toolConfig and generationConfig. Vertex AI's own paths take the same body: see
Vertex AI.
| Model behind the name | What the gateway does |
|---|---|
| Gemini (Google AI Studio) | Sends the call as it is |
| Gemini on Vertex AI | Sends it as it is, on Vertex AI's address |
| OpenAI, Azure OpenAI | Translates the call to Chat Completions, and the answer back |
| Claude (Anthropic, Bedrock, Vertex AI) | Translates it, and the answer back |
| Bedrock's other models (Nova, Llama, Mistral, Cohere) | Translates it |
The model name in the path decides the route; an admin decides in the Control room which provider serves each name.
The two settings¶
| Setting | Environment variable | Python | TypeScript | Value |
|---|---|---|---|---|
| Base address | GOOGLE_GEMINI_BASE_URL |
HttpOptions(base_url=…) |
httpOptions: { baseUrl } |
https://<your-gateway> |
| Key | GEMINI_API_KEY or GOOGLE_API_KEY |
api_key |
apiKey |
Your Vidai key (starts vk_) |
| API version | GOOGLE_GENAI_API_VERSION |
HttpOptions(api_version=…) |
apiVersion |
v1beta (SDK 3.0 and later default to v1, which the gateway doesn't serve) |
Details per SDK: Google GenAI SDK for Python, Google GenAI SDK for TypeScript.
A minimal call¶
from google import genai
from google.genai import types
client = genai.Client(
api_key="vk_…",
http_options=types.HttpOptions(base_url="https://<your-gateway>", api_version="v1beta"),
)
response = client.models.generate_content(
model="gemini-3.5-flash",
contents="Say hello in Dutch.",
)
print(response.text)
import { GoogleGenAI } from "@google/genai";
const ai = new GoogleGenAI({
apiKey: "vk_…",
apiVersion: "v1beta",
httpOptions: { baseUrl: "https://<your-gateway>" },
});
const response = await ai.models.generateContent({
model: "gemini-3.5-flash",
contents: "Say hello in Dutch.",
});
console.log(response.text);
curl "https://<your-gateway>/v1beta/models/gemini-3.5-flash:generateContent" \
-H "x-goog-api-key: $GEMINI_API_KEY" \
-H "content-type: application/json" \
-d '{"contents":[{"role":"user","parts":[{"text":"Say hello in Dutch."}]}]}'
Put gpt-4.1 or claude-sonnet-4.5 (for example) in model and the same code gets a GenerateContentResponse
back: candidates, finishReason and usageMetadata, translated from the other provider's answer.
Streaming¶
for chunk in client.models.generate_content_stream(
model="claude-sonnet-4.5",
contents="Count to five.",
):
if chunk.text:
print(chunk.text, end="")
const stream = await ai.models.generateContentStream({
model: "claude-sonnet-4.5",
contents: "Count to five.",
});
for await (const chunk of stream) {
if (chunk.text) process.stdout.write(chunk.text);
}
curl -N "https://<your-gateway>/v1beta/models/claude-sonnet-4.5:streamGenerateContent?alt=sse" \
-H "x-goog-api-key: $GEMINI_API_KEY" -H "content-type: application/json" \
-d '{"contents":[{"role":"user","parts":[{"text":"Count to five."}]}]}'
A stream from any provider arrives as Gemini's own server-sent events: each data: line is a GenerateContentResponse
chunk. From an OpenAI model, finishReason and usageMetadata arrive on the last two chunks rather than together on
the last one; both arrive. If something fails after the first chunk has reached you, the gateway closes the
connection; a fallback model is only tried before the first byte.
Tool calls¶
Function declarations, toolConfig and function responses work as Google documents them, on every provider: calls
come back as functionCall parts, and you send results as functionResponse parts.
weather = types.FunctionDeclaration(
name="get_weather",
description="Weather for a city",
parameters_json_schema={
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
)
response = client.models.generate_content(
model="gpt-4.1",
contents="Weather in Utrecht?",
config=types.GenerateContentConfig(tools=[types.Tool(function_declarations=[weather])]),
)
print(response.function_calls)
const response = await ai.models.generateContent({
model: "gpt-4.1",
contents: "Weather in Utrecht?",
config: {
tools: [{ functionDeclarations: [{
name: "get_weather",
description: "Weather for a city",
parametersJsonSchema: { type: "object", properties: { city: { type: "string" } }, required: ["city"] },
}] }],
},
});
console.log(response.functionCalls);
On another provider's model:
| You send | What happens |
|---|---|
Function declarations (parameters or parametersJsonSchema) |
Carried both ways as JSON Schema; a name the other provider can't take is renamed on the way and restored in the answer |
allowedFunctionNames, mode ANY / NONE |
Carried |
Mode VALIDATED |
Becomes strict tools where every schema allows it; otherwise reported as toolConfig.functionCallingConfig.mode |
googleSearch |
Becomes web search where the model has it; otherwise dropped and reported as tools.googleSearch |
codeExecution, urlContext, googleSearchRetrieval, fileSearch |
Dropped and reported as tools.<name>; the call goes ahead without them |
A Gemini model gets your schemas too: one that doesn't fit Gemini's own Schema is sent whole as
parametersJsonSchema, so agent frameworks' schemas are accepted.
Reading what the gateway did¶
response = client.models.generate_content(model="claude-sonnet-4.5", contents="Hi")
headers = response.sdk_http_response.headers
print(headers.get("x-vidai-request-id")) # quote this; paste it into Request logs › Request reference
print(headers.get("x-vidai-model")) # the model that answered
print(headers.get("x-vidai-adapted")) # what translation changed or couldn't carry
const response = await ai.models.generateContent({ model: "claude-sonnet-4.5", contents: "Hi" });
const headers = response.sdkHttpResponse?.headers ?? {};
console.log(headers["x-vidai-request-id"], headers["x-vidai-model"], headers["x-vidai-adapted"]);
x-vidai-adapted lists, under the names you sent, what had no equivalent on the provider that answered and was
dropped, clamped or approximated: never a failed call. Tokens you may meet on this endpoint:
| Token | Meaning |
|---|---|
generationConfig.<field> (for example generationConfig.topK) |
A generation setting with no equivalent there |
generationConfig.thinkingConfig.includeThoughts |
The thinking level or budget was carried; showing thoughts was not |
thinkingConfig |
Thinking was dropped: the model can't reason |
safetySettings |
Gemini-only; not sent |
fileData, inlineData.audio, inlineData.video |
Media the other provider can't take |
contents.parts.thought |
Earlier thoughts in the history were not sent on (they were the previous model's) |
tools.<name>, toolConfig.functionCallingConfig.mode |
See Tool calls |
x-vidai-translation |
You asked for a translation profile this key may not choose |
Translation profile. Each key has a Translation between API formats setting: Standard, or Keep what the
other format can't say. If the key has Let apps choose the translation per call, send
x-vidai-translation: default or x-vidai-translation: preserve on one call (in Python,
HttpOptions(headers={...})); x-vidai-translation-profile in the answer says which applied.
Other headers: x-vidai-requested-model, x-vidai-routing-rule, x-vidai-provider-key, x-vidai-fallback,
x-vidai-fallback-from, x-vidai-fallback-attempts and x-vidai-pseudonymised. Cost is never in the answer: find the
call in Request logs.
Errors¶
The gateway's own errors, and a translated provider's errors, come in Google's shape, so the SDK raises its usual
APIError (ClientError for 4xx, ServerError for 5xx). A Gemini provider's own error passes through as Google
wrote it.
{"error":{"code":403,"message":"…","status":"PERMISSION_DENIED"}}
| Status | status |
Common gateway codes (x-vidai-error-code) |
|---|---|---|
| 400, 413 | INVALID_ARGUMENT |
bad_request, provider_override_not_allowed, payload_too_large, embeddings to another provider |
| 401 | UNAUTHENTICATED |
missing_api_key, invalid_api_key, conflicting_auth_headers |
| 403 | UNAUTHENTICATED |
api_key_disabled |
| 403 | PERMISSION_DENIED |
model_not_allowed, denied_by_routing_rule, content_guardrail_blocked |
| 404 | NOT_FOUND |
model_not_found, endpoint_not_found |
| 429 | RESOURCE_EXHAUSTED |
rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded |
| 502, 503 | UNAVAILABLE |
upstream_unreachable, upstream_closed, fallback_exhausted, ai_traffic_suspended |
| 504 | DEADLINE_EXCEEDED |
upstream_timeout |
from google.genai import errors
try:
client.models.generate_content(model="gemini-3.5-flash", contents="Hi")
except errors.APIError as e:
print(e.code, e.status)
print(e.response.headers.get("x-vidai-error-code"), e.response.headers.get("x-vidai-request-id"))
import { ApiError } from "@google/genai";
try {
await ai.models.generateContent({ model: "gemini-3.5-flash", contents: "Hi" });
} catch (e) {
if (e instanceof ApiError) console.log(e.status, e.message);
}
A prompt the other provider blocks comes back as a SAFETY finish, not an empty answer; a Claude refusal comes back
the same way. A key that's switched off carries a short reason with api_key_disabled (for example
agent_suspended); treat the list of reasons as open-ended.
Model names¶
Send the names your organisation approved, as listed under Models in the Control room. client.models.list()
(ai.models.list()) calls GET /v1beta/models, which the gateway answers with exactly the models your key may call,
in Gemini's shape. x-vidai-model says which model really answered. See Model names.
What goes through the gateway¶
| Feature | Gemini (AI Studio, Vertex AI) | OpenAI, Azure, Claude, Bedrock |
|---|---|---|
Text, multi-turn, systemInstruction |
As sent | Translated |
| Streaming | As sent | Translated to Gemini chunks |
| Function calling | As sent (schemas fixed up for Gemini) | Translated |
Structured output (responseJsonSchema, responseSchema) |
As sent | Translated |
Images (inlineData) |
As sent | Translated |
| Audio input | As sent | OpenAI audio models; elsewhere reported |
Video, documents by URL (fileData) |
As sent | Reported, not sent |
Thinking (thinkingConfig) |
As sent | Mapped to the model's reasoning setting; dropped and reported on a model that can't reason |
Spoken replies (speechConfig) |
As sent | Nearest OpenAI voice, reported |
safetySettings |
As sent | Reported, not sent |
Check it works in a minute¶
curl -s -i "https://<your-gateway>/v1beta/models/gemini-3.5-flash:generateContent" \
-H "x-goog-api-key: $GEMINI_API_KEY" -H "content-type: application/json" \
-d '{"contents":[{"parts":[{"text":"ping"}]}],"generationConfig":{"maxOutputTokens":5}}' | grep -i -e "^HTTP" -e "x-vidai-model"
A 200 and an x-vidai-model line mean the address and key are right. The call shows in Request logs within
seconds.
Good to know¶
- If every call fails with a 404 with an empty body (no
x-vidai-error-code) although the model name is right, the SDK is probably calling/v1/models/…:generateContent: set the API version tov1beta(above). - Claude's
529 overloadedarrives as503 UNAVAILABLE, a status Google's SDKs know. - Gemini 3's thought signatures pass through as Gemini wrote them on a Gemini model. When a GPT or Claude model answers, there are none to keep.
- Image URLs work best as inline bytes when the model may be another provider's.
- The gateway's own 429 has no
Retry-Afterheader; back off on your side. If you set the SDK'sretry_options, keep attempts low: a fallback chain may already have tried another model before you see an error (Fallback and retries).