Skip to content

LangChain: using the VIDAI Control Plane as the backend

LangChain's provider integrations (langchain-openai, langchain-anthropic, langchain-google-genai) wrap the official vendor SDKs and forward base_url through. Two of the three work via pure environment variables.

TL;DR

export OPENAI_API_BASE=https://your-vidaiserver.example.com/v1
export OPENAI_API_KEY=sk-your-vidai-key
export ANTHROPIC_API_URL=https://your-vidaiserver.example.com
export ANTHROPIC_API_KEY=sk-your-vidai-key
export GOOGLE_API_KEY=sk-your-vidai-key

from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic
from langchain_google_genai import ChatGoogleGenerativeAI

# Env vars alone; no kwargs needed
chat_oai = ChatOpenAI(model="gpt-4o-mini")
chat_ant = ChatAnthropic(model="claude-haiku-4-5")

# Gemini needs base_url on the constructor (no env var hook today)
chat_gem = ChatGoogleGenerativeAI(
    model="gemini-2.5-flash",
    base_url="https://your-vidaiserver.example.com",
)

Prerequisites

  • langchain-openai >= 1.1, langchain-anthropic >= 1.0, langchain-google-genai >= 4.2 (only install the ones you use)
  • The control plane URL and an API key

New project

  1. Install the integrations you need: 
    pip install langchain-openai langchain-anthropic langchain-google-genai
  2. Set env vars (see TL;DR above).
  3. Use LangChain normally: 
    from langchain_openai import ChatOpenAI
chat = ChatOpenAI(model="gpt-4o-mini")
resp = chat.invoke("summarize LangGraph in one sentence")
print(resp.content)

Existing project: migrating to the VIDAI Control Plane

Change env vars. That's almost always the entire migration.

ChatOpenAI

# Before
export OPENAI_API_KEY=sk-openai-...

# After
export OPENAI_API_BASE=https://your-vidaiserver.example.com/v1
export OPENAI_API_KEY=sk-your-vidai-key
No Python changes.

ChatAnthropic

# Before
export ANTHROPIC_API_KEY=sk-ant-...

# After
export ANTHROPIC_API_URL=https://your-vidaiserver.example.com
export ANTHROPIC_API_KEY=sk-your-vidai-key
No Python changes.

ChatGoogleGenerativeAI

No env var for base_url is read by this wrapper. You need to pass base_url= as a constructor kwarg. If you have many call sites, wrap it:

# llm_factory.py
import os
from langchain_google_genai import ChatGoogleGenerativeAI

def gemini(model: str, **kw) -> ChatGoogleGenerativeAI:
    return ChatGoogleGenerativeAI(
        model=model,
        base_url=os.environ["VIDAI_SERVER_URL"],
        **kw,
    )
Then every call site: 
from llm_factory import gemini
chat = gemini("gemini-2.5-flash")

Streaming

Unchanged: .stream() yields AIMessageChunks; the + reducer produces the final merged message:

chunks = list(chat.stream("count to five"))
merged = chunks[0]
for c in chunks[1:]:
    merged = merged + c
print(merged.content)

Tool binding

from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get current weather for a city."""
    return f"{city}: 20C"

bound = chat.bind_tools([get_weather])
resp = bound.invoke("Weather in Paris?")
# resp.tool_calls is a list of {name, args, id}
print(resp.tool_calls)

Structured output

from pydantic import BaseModel, Field

class WeatherReport(BaseModel):
    city: str = Field(description="city name")
    temperature_c: int = Field(description="temperature in celsius")

structured = chat.with_structured_output(WeatherReport)
result = structured.invoke("Report weather for Paris, 20C")
print(result.city, result.temperature_c)

Cross-provider routing

Ops can register a model that routes to any supported upstream. The LangChain side is unchanged; you just name the model. See the cross-provider matrix in Client integrations for which (LangChain wrapper × upstream) combinations are supported.

Reading the control plane's response headers

LangChain's AIMessage doesn't expose the underlying HTTP response headers directly, so the x-vidai-* routing headers aren't readable from a vanilla chat.invoke(). If you need the routing trail for debug dashboards, the cleanest path is to call the underlying vendor SDK for those specific debug requests (see e.g. openai-sdk.md "Reading the control plane's response headers").

Some versions of the LangChain wrappers populate response_metadata on the returned AIMessage with provider metadata that may include vidai headers; check what your version does:

resp = chat.invoke("hi")
print(resp.response_metadata)

See routing-and-headers.md for what the headers mean.

Verify it works: probe

from langchain_openai import ChatOpenAI
chat = ChatOpenAI(
    model="gpt-4o-mini",
    base_url="https://your-vidaiserver.example.com/v1",
    api_key="sk-your-vidai-key",
)
print(chat.invoke("say: ok").content)

If this prints a response, your setup is correct. If it raises an auth error, the key is wrong. If it raises NotFoundError, the model isn't registered in the control plane for your key.