Skip to content

Instructor

Instructor turns a model's answer into a Pydantic object and asks again when validation fails. Point the client it wraps at the gateway and every structured call, retries included, is checked against your policies, charged to the right team and recorded in Request logs. The same response_model works on Claude and Gemini as well as GPT.

What it is

Instructor is a Python library that wraps an OpenAI or Anthropic client and adds response_model to its create call. It sends ordinary Chat Completions or Messages requests, which the gateway accepts and translates to every provider it serves.

The two settings

Instructor takes no address of its own: you set it on the client it wraps.

Setting Wrapping an OpenAI client Wrapping an Anthropic client Value
Base address base_url (or OPENAI_BASE_URL) base_url (or ANTHROPIC_BASE_URL) https://<your-gateway>/v1 for OpenAI; https://<your-gateway> for Anthropic (the SDK adds /v1/messages)
Key api_key (or OPENAI_API_KEY) api_key (or ANTHROPIC_API_KEY) Your Vidai key (starts vk_), never a provider's key

The gateway's address, and the base URL for each kind of SDK, are under Connect your app on the Keys page and in System › Settings › Gateway. You get the key from Keys › Issue a key; its secret is shown once.

A minimal call

import instructor
from openai import OpenAI
from pydantic import BaseModel

client = instructor.from_openai(
    OpenAI(base_url="https://<your-gateway>/v1", api_key="vk_…")
)

class Ticket(BaseModel):
    priority: str
    reason: str

ticket = client.chat.completions.create(
    model="gpt-4o-mini",
    response_model=Ticket,
    messages=[{"role": "user", "content": "The printer on floor 3 is on fire."}],
)
print(ticket)
import instructor

client = instructor.from_provider(
    "openai/gpt-4o-mini",
    base_url="https://<your-gateway>/v1",
    api_key="vk_…",
)
ticket = client.chat.completions.create(
    response_model=Ticket,
    messages=[{"role": "user", "content": "The printer on floor 3 is on fire."}],
)
import instructor
from anthropic import Anthropic

client = instructor.from_anthropic(
    Anthropic(base_url="https://<your-gateway>", api_key="vk_…")
)
ticket = client.messages.create(
    model="claude-sonnet-4.5",
    max_tokens=500,
    response_model=Ticket,
    messages=[{"role": "user", "content": "The printer on floor 3 is on fire."}],
)

Through the OpenAI client, put claude-sonnet-4.5 or gemini-3.5-flash in model (for example) and the same code works: the gateway translates the request and the answer, tool calls included. The Anthropic client can likewise call GPT or Gemini models.

instructor.from_provider("openai/…", base_url=…) passes base_url to the OpenAI client it builds. For "anthropic/…" it doesn't: use from_anthropic as above, or set ANTHROPIC_BASE_URL before you start.

Streaming

create_partial streams one object as it fills in; create_iterable streams a list of objects one by one.

for partial in client.chat.completions.create_partial(
    model="gpt-4o-mini",
    response_model=Ticket,
    messages=[{"role": "user", "content": "The printer on floor 3 is on fire."}],
):
    print(partial)

A stream from Claude or Gemini arrives in the format of the client you wrapped; Vidai records the tokens and the cost either way.

Tool calls

Instructor's default mode (Mode.TOOLS) asks for the object as a tool call. Tool calls work on every provider the gateway translates to, so the default suits GPT, Claude and Gemini alike. Mode.JSON and Mode.TOOLS_STRICT send response_format, which the gateway also carries to Claude and Gemini models.

Leave out Mode.RESPONSES_TOOLS: it uses the Responses API, which the gateway sends to OpenAI and Azure OpenAI models only.

Reading what the gateway did

create_with_completion returns the raw completion beside your object. For the gateway's headers, call through the wrapped SDK's with_raw_response (see OpenAI SDK for Python), or look the call up in Request logs: filter on Key, Model or Path. x-vidai-request-id, pasted into Request logs › Request reference, finds one call exactly. Cost is never in the answer.

Errors

Errors come from the SDK you wrapped, so the exceptions are that SDK's own (openai.RateLimitError, anthropic.PermissionDeniedError and so on). Match on the x-vidai-error-code header of e.response: codes such as invalid_api_key, model_not_allowed, content_guardrail_blocked or rate_limit_exceeded are stable. The full list is in Headers and errors.

A validation failure after all retries raises Instructor's InstructorRetryException; it isn't a gateway error.

Model names

Send the names your organisation approved under Models in the Control room. models.list() on the OpenAI client you wrapped returns exactly the models your key may call.

What goes through the gateway

Feature Through the gateway
create, create_partial, create_iterable, create_with_completion in TOOLS, JSON, MD_JSON or TOOLS_STRICT mode Yes, to OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models
Mode.RESPONSES_TOOLS and other Responses API modes OpenAI and Azure OpenAI models only; another provider gets a clear 400
Images in the messages Yes, as for chat

Check it works in a minute

python -c "
import instructor; from openai import OpenAI; from pydantic import BaseModel
class R(BaseModel): reply: str
c = instructor.from_openai(OpenAI())  # reads OPENAI_BASE_URL and OPENAI_API_KEY
print(c.chat.completions.create(model='gpt-4o-mini', response_model=R, messages=[{'role':'user','content':'Reply with ok'}]).reply)"

A short answer means the address and key are right. The call shows in Request logs within seconds.

Good to know

  • max_retries on create is Instructor's validation retry: each retry is a new call, with its own row and cost in Request logs. max_retries on from_provider is the SDK's network retry instead.
  • The wrapped SDK retries twice by default, and a fallback chain may already have tried another model before you see an error; build the client with max_retries=0 (or 1) if you don't want both.
  • Instructor's own docs: OpenAI-compatible servers.