Pydantic AI¶
Your Pydantic AI agents keep their types, tools and structured output. Give the provider two settings and every model call in a run goes through the gateway: checked against your policies, charged to the right team and recorded in Request logs, on Claude, Gemini or OpenAI models.
What it is¶
Pydantic AI is a Python agent framework from the Pydantic team. A model class (the API format) uses a provider (where to send the call and with which key). Three combinations work with the gateway:
| Model class | Provider | Format the gateway receives | Reaches |
|---|---|---|---|
OpenAIChatModel |
OpenAIProvider |
OpenAI Chat Completions, /v1/chat/completions |
OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models |
AnthropicModel |
AnthropicProvider |
Anthropic Messages, /v1/messages |
The same models, translated both ways |
GoogleModel |
GoogleProvider |
Gemini, /v1beta/models/…:generateContent |
The same models, translated both ways |
OpenAIChatModel is the one to start with. OpenAIModel is its old name, removed in Pydantic AI 2.
The two settings¶
| Provider | Base address | Key | Environment variables |
|---|---|---|---|
OpenAIProvider |
base_url="https://<your-gateway>/v1" |
api_key="vk_…" |
OPENAI_BASE_URL, OPENAI_API_KEY |
AnthropicProvider |
base_url="https://<your-gateway>" (no /v1) |
api_key="vk_…" |
ANTHROPIC_API_KEY (pass base_url in code) |
GoogleProvider |
base_url="https://<your-gateway>" (no /v1beta) |
api_key="vk_…" |
GOOGLE_API_KEY (pass base_url in code) |
The key is your Vidai key (starts vk_), never a provider's key. The OpenAI provider sends it as
Authorization: Bearer, the Anthropic provider as x-api-key, the Google provider as x-goog-api-key; the gateway
accepts all three.
The gateway's address, and the base URL for each kind of SDK, are under Connect your app on the Keys page and in System › Settings › Gateway. You get the key from Keys › Issue a key; its secret is shown once.
A minimal call¶
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
model = OpenAIChatModel(
"gpt-4o-mini",
provider=OpenAIProvider(base_url="https://<your-gateway>/v1", api_key="vk_…"),
)
agent = Agent(model)
print(agent.run_sync("Say hello in Dutch.").output)
The same code calls Claude or Gemini: put a model name such as claude-sonnet-4.5 in the model class. The gateway
translates the request and the answer, tools and structured output included.
With OPENAI_BASE_URL and OPENAI_API_KEY set, the short form works too: Agent("openai-chat:gpt-4o-mini"). Use
openai-chat:, not openai:: the bare prefix picks the Responses API, which the gateway sends to OpenAI models only.
The other two formats:
from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai.providers.anthropic import AnthropicProvider
model = AnthropicModel(
"claude-sonnet-4.5",
provider=AnthropicProvider(base_url="https://<your-gateway>", api_key="vk_…"),
)
from pydantic_ai.models.google import GoogleModel
from pydantic_ai.providers.google import GoogleProvider
model = GoogleModel(
"gemini-3.5-flash",
provider=GoogleProvider(base_url="https://<your-gateway>", api_key="vk_…"),
)
Streaming¶
async with agent.run_stream("Count to five.") as result:
async for text in result.stream_text(delta=True):
print(text, end="")
A stream from another provider arrives in the format the model class speaks. Vidai records the tokens and the cost either way.
Tool calls¶
Register tools with @agent.tool (or @agent.tool_plain when the tool needs no run context). Each round trip is a
call through the gateway, on any provider it translates to.
agent = Agent(model)
@agent.tool_plain
def get_weather(city: str) -> str:
"""Weather for a city."""
return f"Sunny in {city}"
print(agent.run_sync("Weather in Utrecht?").output)
Structured output (output_type= a Pydantic model) is carried the same way.
Reading what the gateway did¶
Pydantic AI doesn't hand you the headers of a successful answer. Find the calls in Request logs: filter by Key or Model; each step of a run is its own call. The model that answered, any routing rule or fallback, and the cost are there. On an error, the headers are on the exception (next section).
Errors¶
A refusal from the gateway raises ModelHTTPError with the HTTP status, the error body (written in the format the
model class speaks) and the response headers. Match on the x-vidai-error-code header rather than the message:
codes are stable.
from pydantic_ai.exceptions import ModelHTTPError
try:
agent.run_sync("Hi")
except ModelHTTPError as e:
headers = e.headers or {}
print(e.status_code, headers.get("x-vidai-error-code"))
print(headers.get("x-vidai-request-id")) # paste into Request logs › Request reference
| Status | Common gateway codes |
|---|---|
| 401 | missing_api_key, invalid_api_key, conflicting_auth_headers |
| 403 | api_key_disabled, model_not_allowed, denied_by_routing_rule, content_guardrail_blocked |
| 404 | model_not_found, endpoint_not_found |
| 429 | rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded |
| 5xx | ai_traffic_suspended, fallback_exhausted, upstream_timeout, upstream_unreachable |
ModelRateLimitError catches the 429s on their own; it's a ModelAPIError, not a ModelHTTPError, so catch it
separately (or catch ModelAPIError) if you want both. The full list is in
Headers and errors.
Model names¶
Send the names your organisation has approved, as listed under Models in the Control room (for example
gpt-4o-mini, claude-sonnet-4.5, gemini-3.5-flash). GET /v1/models with your key returns exactly the models it
may call. See Model names.
What goes through the gateway¶
| Feature | Through the gateway |
|---|---|
Agent runs, streaming, tools, structured output, images, with OpenAIChatModel, AnthropicModel or GoogleModel |
Yes, to OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models |
OpenAIResponsesModel (and the openai: prefix) |
OpenAI and Azure OpenAI models only; another provider gets a clear 400 |
| Embeddings with OpenAI's embedding models | Yes, to OpenAI and Azure OpenAI only |
Check it works in a minute¶
export OPENAI_BASE_URL="https://<your-gateway>/v1" OPENAI_API_KEY="vk_…"
python -c "from pydantic_ai import Agent; print(Agent('openai-chat:gpt-4o-mini').run_sync('ping').output)"
A short answer means the address and key are right. The call shows in Request logs within seconds.
Good to know¶
- The SDKs under each provider retry on their own. A fallback chain may already have tried another model before you see
an error; to turn the SDK's retries off, build the client yourself (for example
AsyncOpenAI(..., max_retries=0)) and pass it asOpenAIProvider(openai_client=...). - The gateway's own 429 has no
Retry-Afterheader; back off on your side. - Anthropic's schema-defined tools (bash, text editor) work on Claude models only; prompt-caching markers are dropped on other providers' models.