Skip to content

LangChain

Your chains, agents and tools keep calling model.invoke(...). Give the chat model two settings and every call is governed: checked against your policies, charged to the right team and recorded in Request logs. Any of LangChain's OpenAI, Anthropic or Google chat models can reach any provider's models, because the gateway translates between them.

What it is

LangChain's chat model packages for Python: langchain-openai (ChatOpenAI), langchain-anthropic (ChatAnthropic) and langchain-google-genai (ChatGoogleGenerativeAI). Each wraps its provider's official SDK, so each speaks one of the formats the gateway accepts: OpenAI's at /v1, Anthropic's at /v1/messages, Gemini's at /v1beta/models/…. Pick the one you already use; ChatOpenAI is the simplest.

The two settings

Chat model Base address Key Value of the base address
ChatOpenAI base_url= or OPENAI_BASE_URL (OPENAI_API_BASE is also read) api_key= or OPENAI_API_KEY https://<your-gateway>/v1
ChatAnthropic base_url= or ANTHROPIC_BASE_URL (ANTHROPIC_API_URL is also read) api_key= or ANTHROPIC_API_KEY https://<your-gateway> (the SDK adds /v1/messages)
ChatGoogleGenerativeAI base_url= api_key= or GOOGLE_API_KEY https://<your-gateway> (the SDK adds /v1beta/models/…)

The key is always a Vidai key (starts vk_), never a provider's key; each SDK sends it in a header the gateway accepts (Authorization: Bearer, x-api-key or x-goog-api-key). You get the key from Keys › Issue a key, or, for an agent, from the agent's Keys section under Applications & agents; its secret is shown once. The gateway's address and the base URL for each kind of SDK are under Connect your app on the Keys page and in System › Settings › Gateway.

export OPENAI_BASE_URL="https://<your-gateway>/v1"
export OPENAI_API_KEY="vk_…"

A minimal call

from langchain_openai import ChatOpenAI

model = ChatOpenAI(model="gpt-4o-mini")  # reads OPENAI_BASE_URL and OPENAI_API_KEY
# or: ChatOpenAI(model="gpt-4o-mini", base_url="https://<your-gateway>/v1", api_key="vk_…")
print(model.invoke("Say hello in Dutch.").content)
from langchain_anthropic import ChatAnthropic

model = ChatAnthropic(model="claude-sonnet-4.5", base_url="https://<your-gateway>", api_key="vk_…")
print(model.invoke("Say hello in Dutch.").content)
from langchain_google_genai import ChatGoogleGenerativeAI

model = ChatGoogleGenerativeAI(model="gemini-3.5-flash", base_url="https://<your-gateway>", api_key="vk_…")
print(model.invoke("Say hello in Dutch.").content)

Any of them calls any provider: ChatOpenAI(model="claude-sonnet-4.5"), ChatAnthropic(model="gpt-4o-mini") or ChatGoogleGenerativeAI(model="gpt-4o-mini") all work. The gateway translates the request for that provider and the answer back into the shape the chat model expects, streams and tool calls included.

Streaming

for chunk in model.stream("Count to five."):
    print(chunk.content, end="")

A stream from another provider arrives in the chat model's own format. With a custom base address ChatOpenAI doesn't ask for the usage chunk; pass stream_usage=True to get token counts in usage_metadata. Vidai records the tokens and the cost either way.

Tool calls

bind_tools and with_structured_output work on every provider the gateway translates to:

from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Weather for a city."""
    return f"Sunny in {city}"

reply = ChatOpenAI(model="claude-sonnet-4.5").bind_tools([get_weather]).invoke("Weather in Utrecht?")
print(reply.tool_calls)

Provider-hosted tools are different: OpenAI's built-in tools (web search, file search, code interpreter) switch ChatOpenAI to the Responses API, which reaches only OpenAI and Azure OpenAI models.

Reading what the gateway did

ChatOpenAI can hand you the response headers in the message's response_metadata:

model = ChatOpenAI(model="gpt-4o-mini", include_response_headers=True)
reply = model.invoke("Hi")
headers = reply.response_metadata["headers"]
print(headers.get("x-vidai-request-id"))  # quote this; paste it into Request logs › Request reference
print(headers.get("x-vidai-model"))       # the model that answered
print(headers.get("x-vidai-adapted"))     # what translation changed, if anything

ChatAnthropic and ChatGoogleGenerativeAI don't pass the headers on: find their calls in Request logs (filter Key, Model or Path). Cost is never in the answer; it's in Request logs.

Errors

The chat models raise their SDK's exceptions: the gateway writes each error in the caller's own format. Match on the x-vidai-error-code header where the exception carries the response (ChatOpenAI and ChatAnthropic do).

Status ChatOpenAI / ChatAnthropic exception Common gateway codes
400 BadRequestError bad_request, an endpoint this provider doesn't serve
401 AuthenticationError missing_api_key, invalid_api_key, conflicting_auth_headers
403 PermissionDeniedError api_key_disabled, model_not_allowed, denied_by_routing_rule, content_guardrail_blocked
404 NotFoundError model_not_found, endpoint_not_found
429 RateLimitError rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded
5xx InternalServerError ai_traffic_suspended, fallback_exhausted, upstream_timeout, upstream_unreachable
import openai

try:
    ChatOpenAI(model="gpt-4o-mini").invoke("Hi")
except openai.APIStatusError as e:
    print(e.status_code, e.response.headers.get("x-vidai-error-code"))

ChatGoogleGenerativeAI raises Google's errors with the status; find the call by status and time in Request logs.

Model names

Send the names your organisation has approved, as listed under Models in the Control room. GET /v1/models on the gateway returns exactly the models your key may call. An alias set by an admin works like any other name; the x-vidai-model header says which model really answered.

What goes through the gateway

Feature Through the gateway
invoke, stream, batch, tools, structured output, vision, on any of the three chat models Yes, to OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models
ChatOpenAI on the Responses API (use_responses_api=True, reasoning=, built-in tools, some models' defaults) Yes, to OpenAI and Azure OpenAI only; another provider gets a clear 400
OpenAIEmbeddings(base_url=…, api_key=…) Yes, to OpenAI and Azure OpenAI embedding models only
LangSmith tracing No: it goes to LangSmith directly, not through the gateway

Check it works in a minute

python -c "from langchain_openai import ChatOpenAI; print(ChatOpenAI(model='gpt-4o-mini', max_tokens=5).invoke('ping').content)"

A short answer means the address and key are right. The call shows in Request logs within seconds.

Good to know

  • Each chat model retries on its own: ChatOpenAI and ChatAnthropic twice by default, ChatGoogleGenerativeAI six times. A fallback chain may already have tried another model first; set max_retries=0 on ChatOpenAI and ChatAnthropic, and max_retries=1 on ChatGoogleGenerativeAI (there, 1 means no retry), if you don't want both.
  • ChatOpenAI moves to the Responses API by itself when a call uses a Responses-only feature (reasoning=, built-in tools, use_previous_response_id) or a Responses-only model. Set use_responses_api=False to keep it on Chat Completions when the model isn't OpenAI's.
  • ChatGoogleGenerativeAI with vertexai=True (or GOOGLE_GENAI_USE_VERTEXAI=true) expects Google credentials, which the gateway doesn't take; leave it on the Gemini API with the Vidai key.
  • The gateway's own 429 has no Retry-After header; back off on your side.
  • Agents built with LangGraph use these same chat models: see LangGraph.