LangChain¶
Your chains, agents and tools keep calling model.invoke(...). Give the chat model two settings and every call is
governed: checked against your policies, charged to the right team and recorded in Request logs. Any of LangChain's
OpenAI, Anthropic or Google chat models can reach any provider's models, because the gateway translates between them.
What it is¶
LangChain's chat model packages for Python: langchain-openai (ChatOpenAI), langchain-anthropic (ChatAnthropic)
and langchain-google-genai (ChatGoogleGenerativeAI). Each wraps its provider's official SDK, so each speaks one of
the formats the gateway accepts: OpenAI's at /v1, Anthropic's at /v1/messages, Gemini's at /v1beta/models/….
Pick the one you already use; ChatOpenAI is the simplest.
The two settings¶
| Chat model | Base address | Key | Value of the base address |
|---|---|---|---|
ChatOpenAI |
base_url= or OPENAI_BASE_URL (OPENAI_API_BASE is also read) |
api_key= or OPENAI_API_KEY |
https://<your-gateway>/v1 |
ChatAnthropic |
base_url= or ANTHROPIC_BASE_URL (ANTHROPIC_API_URL is also read) |
api_key= or ANTHROPIC_API_KEY |
https://<your-gateway> (the SDK adds /v1/messages) |
ChatGoogleGenerativeAI |
base_url= |
api_key= or GOOGLE_API_KEY |
https://<your-gateway> (the SDK adds /v1beta/models/…) |
The key is always a Vidai key (starts vk_), never a provider's key; each SDK sends it in a header the gateway
accepts (Authorization: Bearer, x-api-key or x-goog-api-key). You get the key from Keys › Issue a key, or, for
an agent, from the agent's Keys section under Applications & agents; its secret is shown once. The gateway's
address and the base URL for each kind of SDK are under Connect your app on the Keys page and in System › Settings ›
Gateway.
export OPENAI_BASE_URL="https://<your-gateway>/v1"
export OPENAI_API_KEY="vk_…"
A minimal call¶
from langchain_openai import ChatOpenAI
model = ChatOpenAI(model="gpt-4o-mini") # reads OPENAI_BASE_URL and OPENAI_API_KEY
# or: ChatOpenAI(model="gpt-4o-mini", base_url="https://<your-gateway>/v1", api_key="vk_…")
print(model.invoke("Say hello in Dutch.").content)
from langchain_anthropic import ChatAnthropic
model = ChatAnthropic(model="claude-sonnet-4.5", base_url="https://<your-gateway>", api_key="vk_…")
print(model.invoke("Say hello in Dutch.").content)
from langchain_google_genai import ChatGoogleGenerativeAI
model = ChatGoogleGenerativeAI(model="gemini-3.5-flash", base_url="https://<your-gateway>", api_key="vk_…")
print(model.invoke("Say hello in Dutch.").content)
Any of them calls any provider: ChatOpenAI(model="claude-sonnet-4.5"), ChatAnthropic(model="gpt-4o-mini") or
ChatGoogleGenerativeAI(model="gpt-4o-mini") all work. The gateway translates the request for that provider and the
answer back into the shape the chat model expects, streams and tool calls included.
Streaming¶
for chunk in model.stream("Count to five."):
print(chunk.content, end="")
A stream from another provider arrives in the chat model's own format. With a custom base address ChatOpenAI doesn't
ask for the usage chunk; pass stream_usage=True to get token counts in usage_metadata. Vidai records the tokens
and the cost either way.
Tool calls¶
bind_tools and with_structured_output work on every provider the gateway translates to:
from langchain_core.tools import tool
@tool
def get_weather(city: str) -> str:
"""Weather for a city."""
return f"Sunny in {city}"
reply = ChatOpenAI(model="claude-sonnet-4.5").bind_tools([get_weather]).invoke("Weather in Utrecht?")
print(reply.tool_calls)
Provider-hosted tools are different: OpenAI's built-in tools (web search, file search, code interpreter) switch
ChatOpenAI to the Responses API, which reaches only OpenAI and Azure OpenAI models.
Reading what the gateway did¶
ChatOpenAI can hand you the response headers in the message's response_metadata:
model = ChatOpenAI(model="gpt-4o-mini", include_response_headers=True)
reply = model.invoke("Hi")
headers = reply.response_metadata["headers"]
print(headers.get("x-vidai-request-id")) # quote this; paste it into Request logs › Request reference
print(headers.get("x-vidai-model")) # the model that answered
print(headers.get("x-vidai-adapted")) # what translation changed, if anything
ChatAnthropic and ChatGoogleGenerativeAI don't pass the headers on: find their calls in Request logs (filter
Key, Model or Path). Cost is never in the answer; it's in Request logs.
Errors¶
The chat models raise their SDK's exceptions: the gateway writes each error in the caller's own format. Match on the
x-vidai-error-code header where the exception carries the response (ChatOpenAI and ChatAnthropic do).
| Status | ChatOpenAI / ChatAnthropic exception |
Common gateway codes |
|---|---|---|
| 400 | BadRequestError |
bad_request, an endpoint this provider doesn't serve |
| 401 | AuthenticationError |
missing_api_key, invalid_api_key, conflicting_auth_headers |
| 403 | PermissionDeniedError |
api_key_disabled, model_not_allowed, denied_by_routing_rule, content_guardrail_blocked |
| 404 | NotFoundError |
model_not_found, endpoint_not_found |
| 429 | RateLimitError |
rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded |
| 5xx | InternalServerError |
ai_traffic_suspended, fallback_exhausted, upstream_timeout, upstream_unreachable |
import openai
try:
ChatOpenAI(model="gpt-4o-mini").invoke("Hi")
except openai.APIStatusError as e:
print(e.status_code, e.response.headers.get("x-vidai-error-code"))
ChatGoogleGenerativeAI raises Google's errors with the status; find the call by status and time in Request logs.
Model names¶
Send the names your organisation has approved, as listed under Models in the Control room. GET /v1/models on the
gateway returns exactly the models your key may call. An alias set by an admin works like any other name; the
x-vidai-model header says which model really answered.
What goes through the gateway¶
| Feature | Through the gateway |
|---|---|
invoke, stream, batch, tools, structured output, vision, on any of the three chat models |
Yes, to OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models |
ChatOpenAI on the Responses API (use_responses_api=True, reasoning=, built-in tools, some models' defaults) |
Yes, to OpenAI and Azure OpenAI only; another provider gets a clear 400 |
OpenAIEmbeddings(base_url=…, api_key=…) |
Yes, to OpenAI and Azure OpenAI embedding models only |
| LangSmith tracing | No: it goes to LangSmith directly, not through the gateway |
Check it works in a minute¶
python -c "from langchain_openai import ChatOpenAI; print(ChatOpenAI(model='gpt-4o-mini', max_tokens=5).invoke('ping').content)"
A short answer means the address and key are right. The call shows in Request logs within seconds.
Good to know¶
- Each chat model retries on its own:
ChatOpenAIandChatAnthropictwice by default,ChatGoogleGenerativeAIsix times. A fallback chain may already have tried another model first; setmax_retries=0onChatOpenAIandChatAnthropic, andmax_retries=1onChatGoogleGenerativeAI(there,1means no retry), if you don't want both. ChatOpenAImoves to the Responses API by itself when a call uses a Responses-only feature (reasoning=, built-in tools,use_previous_response_id) or a Responses-only model. Setuse_responses_api=Falseto keep it on Chat Completions when the model isn't OpenAI's.ChatGoogleGenerativeAIwithvertexai=True(orGOOGLE_GENAI_USE_VERTEXAI=true) expects Google credentials, which the gateway doesn't take; leave it on the Gemini API with the Vidai key.- The gateway's own 429 has no
Retry-Afterheader; back off on your side. - Agents built with LangGraph use these same chat models: see LangGraph.