Skip to content

Pydantic AI

Your Pydantic AI agents keep their types, tools and structured output. Give the provider two settings and every model call in a run goes through the gateway: checked against your policies, charged to the right team and recorded in Request logs, on Claude, Gemini or OpenAI models.

What it is

Pydantic AI is a Python agent framework from the Pydantic team. A model class (the API format) uses a provider (where to send the call and with which key). Three combinations work with the gateway:

Model class Provider Format the gateway receives Reaches
OpenAIChatModel OpenAIProvider OpenAI Chat Completions, /v1/chat/completions OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models
AnthropicModel AnthropicProvider Anthropic Messages, /v1/messages The same models, translated both ways
GoogleModel GoogleProvider Gemini, /v1beta/models/…:generateContent The same models, translated both ways

OpenAIChatModel is the one to start with. OpenAIModel is its old name, removed in Pydantic AI 2.

The two settings

Provider Base address Key Environment variables
OpenAIProvider base_url="https://<your-gateway>/v1" api_key="vk_…" OPENAI_BASE_URL, OPENAI_API_KEY
AnthropicProvider base_url="https://<your-gateway>" (no /v1) api_key="vk_…" ANTHROPIC_API_KEY (pass base_url in code)
GoogleProvider base_url="https://<your-gateway>" (no /v1beta) api_key="vk_…" GOOGLE_API_KEY (pass base_url in code)

The key is your Vidai key (starts vk_), never a provider's key. The OpenAI provider sends it as Authorization: Bearer, the Anthropic provider as x-api-key, the Google provider as x-goog-api-key; the gateway accepts all three.

The gateway's address, and the base URL for each kind of SDK, are under Connect your app on the Keys page and in System › Settings › Gateway. You get the key from Keys › Issue a key; its secret is shown once.

A minimal call

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel(
    "gpt-4o-mini",
    provider=OpenAIProvider(base_url="https://<your-gateway>/v1", api_key="vk_…"),
)
agent = Agent(model)
print(agent.run_sync("Say hello in Dutch.").output)

The same code calls Claude or Gemini: put a model name such as claude-sonnet-4.5 in the model class. The gateway translates the request and the answer, tools and structured output included.

With OPENAI_BASE_URL and OPENAI_API_KEY set, the short form works too: Agent("openai-chat:gpt-4o-mini"). Use openai-chat:, not openai:: the bare prefix picks the Responses API, which the gateway sends to OpenAI models only.

The other two formats:

from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai.providers.anthropic import AnthropicProvider

model = AnthropicModel(
    "claude-sonnet-4.5",
    provider=AnthropicProvider(base_url="https://<your-gateway>", api_key="vk_…"),
)
from pydantic_ai.models.google import GoogleModel
from pydantic_ai.providers.google import GoogleProvider

model = GoogleModel(
    "gemini-3.5-flash",
    provider=GoogleProvider(base_url="https://<your-gateway>", api_key="vk_…"),
)

Streaming

async with agent.run_stream("Count to five.") as result:
    async for text in result.stream_text(delta=True):
        print(text, end="")

A stream from another provider arrives in the format the model class speaks. Vidai records the tokens and the cost either way.

Tool calls

Register tools with @agent.tool (or @agent.tool_plain when the tool needs no run context). Each round trip is a call through the gateway, on any provider it translates to.

agent = Agent(model)

@agent.tool_plain
def get_weather(city: str) -> str:
    """Weather for a city."""
    return f"Sunny in {city}"

print(agent.run_sync("Weather in Utrecht?").output)

Structured output (output_type= a Pydantic model) is carried the same way.

Reading what the gateway did

Pydantic AI doesn't hand you the headers of a successful answer. Find the calls in Request logs: filter by Key or Model; each step of a run is its own call. The model that answered, any routing rule or fallback, and the cost are there. On an error, the headers are on the exception (next section).

Errors

A refusal from the gateway raises ModelHTTPError with the HTTP status, the error body (written in the format the model class speaks) and the response headers. Match on the x-vidai-error-code header rather than the message: codes are stable.

from pydantic_ai.exceptions import ModelHTTPError

try:
    agent.run_sync("Hi")
except ModelHTTPError as e:
    headers = e.headers or {}
    print(e.status_code, headers.get("x-vidai-error-code"))
    print(headers.get("x-vidai-request-id"))  # paste into Request logs › Request reference
Status Common gateway codes
401 missing_api_key, invalid_api_key, conflicting_auth_headers
403 api_key_disabled, model_not_allowed, denied_by_routing_rule, content_guardrail_blocked
404 model_not_found, endpoint_not_found
429 rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded
5xx ai_traffic_suspended, fallback_exhausted, upstream_timeout, upstream_unreachable

ModelRateLimitError catches the 429s on their own; it's a ModelAPIError, not a ModelHTTPError, so catch it separately (or catch ModelAPIError) if you want both. The full list is in Headers and errors.

Model names

Send the names your organisation has approved, as listed under Models in the Control room (for example gpt-4o-mini, claude-sonnet-4.5, gemini-3.5-flash). GET /v1/models with your key returns exactly the models it may call. See Model names.

What goes through the gateway

Feature Through the gateway
Agent runs, streaming, tools, structured output, images, with OpenAIChatModel, AnthropicModel or GoogleModel Yes, to OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models
OpenAIResponsesModel (and the openai: prefix) OpenAI and Azure OpenAI models only; another provider gets a clear 400
Embeddings with OpenAI's embedding models Yes, to OpenAI and Azure OpenAI only

Check it works in a minute

export OPENAI_BASE_URL="https://<your-gateway>/v1" OPENAI_API_KEY="vk_…"
python -c "from pydantic_ai import Agent; print(Agent('openai-chat:gpt-4o-mini').run_sync('ping').output)"

A short answer means the address and key are right. The call shows in Request logs within seconds.

Good to know

  • The SDKs under each provider retry on their own. A fallback chain may already have tried another model before you see an error; to turn the SDK's retries off, build the client yourself (for example AsyncOpenAI(..., max_retries=0)) and pass it as OpenAIProvider(openai_client=...).
  • The gateway's own 429 has no Retry-After header; back off on your side.
  • Anthropic's schema-defined tools (bash, text editor) work on Claude models only; prompt-caching markers are dropped on other providers' models.