AutoGen¶
A team of AutoGen agents makes many calls for one task: the planner, the critic, each tool round trip. Give the model client two settings and every one of those calls goes through the gateway: checked against your policies, charged to the key's team and recorded in Request logs, whichever provider's model each agent uses.
What it is¶
AutoGen is Microsoft's Python framework for multi-agent applications: autogen-agentchat for agents and teams, and
autogen-ext for model clients. Agents call models through a model client; this page uses two of them:
OpenAIChatCompletionClient(autogen-ext[openai]): speaks the OpenAI format at/v1/chat/completions. Through the gateway it reaches OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models.AnthropicChatCompletionClient(autogen-ext[anthropic], marked experimental by AutoGen): speaks the Anthropic format at/v1/messages. Through the gateway it reaches the same models, translated both ways.
Microsoft has archived AutoGen and recommends Microsoft Agent Framework for new projects. The settings on this page work for applications already built on AutoGen 0.4 or later.
The two settings¶
| Setting | OpenAIChatCompletionClient |
AnthropicChatCompletionClient |
Value |
|---|---|---|---|
| Base address | base_url |
base_url |
https://<your-gateway>/v1 (OpenAI client) · https://<your-gateway> (Anthropic client) |
| Key | api_key (or OPENAI_API_KEY) |
api_key (or ANTHROPIC_API_KEY) |
Your Vidai key (starts vk_), never a provider's key |
The OpenAI client sends the key as Authorization: Bearer, the Anthropic client as x-api-key; the gateway accepts
both. Always pass base_url in code: for a model name starting claude- or gemini-, the OpenAI client ignores
OPENAI_BASE_URL and sends the call to that provider's own address.
The gateway's address, and the base URL for each kind of SDK, are under Connect your app on the Keys page and in System › Settings › Gateway. You get the key from Keys › Issue a key; its secret is shown once.
One more argument matters: model_info. AutoGen knows the capabilities of OpenAI's own model names. For any other
name (a Claude or Gemini model, or an alias your admin set up) you must pass model_info, or the client refuses to
start with "model_info is required when model name is not a valid OpenAI model".
A minimal call¶
import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_core.models import ModelInfo
from autogen_ext.models.openai import OpenAIChatCompletionClient
model_client = OpenAIChatCompletionClient(
model="claude-sonnet-4.5",
base_url="https://<your-gateway>/v1",
api_key="vk_…",
model_info=ModelInfo(
vision=True, function_calling=True, json_output=True, structured_output=True, family="unknown",
),
)
async def main() -> None:
agent = AssistantAgent("assistant", model_client=model_client)
result = await agent.run(task="Say hello in Dutch.")
print(result.messages[-1].content)
await model_client.close()
asyncio.run(main())
For an OpenAI model name such as gpt-4o-mini you can leave model_info out. Set its fields to what the model really
supports: AutoGen uses them to decide, for example, whether to send images or tools.
To use the Anthropic format instead:
from autogen_ext.models.anthropic import AnthropicChatCompletionClient
model_client = AnthropicChatCompletionClient(
model="claude-sonnet-4.5",
base_url="https://<your-gateway>", # no /v1: the SDK adds /v1/messages
api_key="vk_…",
model_info=ModelInfo(
vision=True, function_calling=True, json_output=True, structured_output=True, family="unknown",
),
)
The same Anthropic client can call gpt-4o-mini or gemini-3.5-flash: the gateway translates the request and the
answer.
Agents in a team can share one client (all their calls are charged to one key) or each have their own: a cheaper model for the planner and a stronger one for the critic, for example. To tell agents apart in Request logs and in cost, give each its own key.
Streaming¶
Set model_client_stream=True on the agent and stream the run:
from autogen_agentchat.ui import Console
agent = AssistantAgent("assistant", model_client=model_client, model_client_stream=True)
await Console(agent.run_stream(task="Count to five."))
A stream from Claude or Gemini arrives in the format the client speaks (OpenAI chunks or Anthropic events). Vidai records the tokens and the cost either way.
Tool calls¶
Pass Python functions in tools; AutoGen describes them to the model and runs the calls. Every round trip is a call
through the gateway, on any provider it translates to.
async def get_weather(city: str) -> str:
"""Weather for a city."""
return f"Sunny in {city}"
agent = AssistantAgent("assistant", model_client=model_client, tools=[get_weather])
await Console(agent.run_stream(task="Weather in Utrecht?"))
The model must have function_calling=True in its model_info.
Reading what the gateway did¶
AutoGen doesn't hand you the response headers. Find the calls in Request logs: filter by Key or Model; each agent turn and tool round trip is its own call. The model that answered, any routing rule or fallback, and the cost are all there. On an error, the headers are on the exception (next section).
Errors¶
The gateway writes its errors in the format the client speaks, so the client raises the SDK's usual exception
(openai.APIStatusError and its subclasses, or anthropic.APIStatusError). Match on the x-vidai-error-code header,
not the message:
import openai
try:
await agent.run(task="Hi")
except openai.APIStatusError as e:
print(e.status_code, e.response.headers.get("x-vidai-error-code"))
print(e.response.headers.get("x-vidai-request-id"))
Common codes: invalid_api_key (401), model_not_allowed, denied_by_routing_rule, content_guardrail_blocked
(403), model_not_found (404), rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded (429),
fallback_exhausted, upstream_timeout (5xx). The full list is in
Headers and errors.
Model names¶
Send the names your organisation has approved, as listed under Models in the Control room (for example
gpt-4o-mini, claude-sonnet-4.5, gemini-3.5-flash). GET /v1/models with your key returns exactly the models it
may call. See Model names.
What goes through the gateway¶
| Feature | Through the gateway |
|---|---|
Agents, teams, streaming, tools, structured output, vision (OpenAIChatCompletionClient) |
Yes, to OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models |
The same with AnthropicChatCompletionClient |
Yes, to the same models, translated both ways |
OpenAIAssistantAgent (Assistants API) |
No: see Assistants API |
Check it works in a minute¶
python -c "import asyncio; from autogen_ext.models.openai import OpenAIChatCompletionClient as C; from autogen_core.models import UserMessage; print(asyncio.run(C(model='gpt-4o-mini', base_url='https://<your-gateway>/v1', api_key='vk_…').create([UserMessage(content='ping', source='user')])).content)"
A short answer means the address and key are right. The call shows in Request logs within seconds.
Good to know¶
- With a model name starting
claude-orgemini-,OpenAIChatCompletionClientsends calls straight to that provider (and readsANTHROPIC_API_KEYorGEMINI_API_KEY) unless you passbase_urlandapi_key: always pass both. - The OpenAI client retries on its own (
max_retries, two by default). A fallback chain may already have tried another model before you see an error; passmax_retries=0if you don't want both. - The gateway's own 429 has no
Retry-Afterheader; back off on your side. - A team can run many turns; a
max_turnsor termination condition keeps one task from spending more than you meant.