Skip to content

AutoGen

A team of AutoGen agents makes many calls for one task: the planner, the critic, each tool round trip. Give the model client two settings and every one of those calls goes through the gateway: checked against your policies, charged to the key's team and recorded in Request logs, whichever provider's model each agent uses.

What it is

AutoGen is Microsoft's Python framework for multi-agent applications: autogen-agentchat for agents and teams, and autogen-ext for model clients. Agents call models through a model client; this page uses two of them:

  • OpenAIChatCompletionClient (autogen-ext[openai]): speaks the OpenAI format at /v1/chat/completions. Through the gateway it reaches OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models.
  • AnthropicChatCompletionClient (autogen-ext[anthropic], marked experimental by AutoGen): speaks the Anthropic format at /v1/messages. Through the gateway it reaches the same models, translated both ways.

Microsoft has archived AutoGen and recommends Microsoft Agent Framework for new projects. The settings on this page work for applications already built on AutoGen 0.4 or later.

The two settings

Setting OpenAIChatCompletionClient AnthropicChatCompletionClient Value
Base address base_url base_url https://<your-gateway>/v1 (OpenAI client) · https://<your-gateway> (Anthropic client)
Key api_key (or OPENAI_API_KEY) api_key (or ANTHROPIC_API_KEY) Your Vidai key (starts vk_), never a provider's key

The OpenAI client sends the key as Authorization: Bearer, the Anthropic client as x-api-key; the gateway accepts both. Always pass base_url in code: for a model name starting claude- or gemini-, the OpenAI client ignores OPENAI_BASE_URL and sends the call to that provider's own address.

The gateway's address, and the base URL for each kind of SDK, are under Connect your app on the Keys page and in System › Settings › Gateway. You get the key from Keys › Issue a key; its secret is shown once.

One more argument matters: model_info. AutoGen knows the capabilities of OpenAI's own model names. For any other name (a Claude or Gemini model, or an alias your admin set up) you must pass model_info, or the client refuses to start with "model_info is required when model name is not a valid OpenAI model".

A minimal call

import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_core.models import ModelInfo
from autogen_ext.models.openai import OpenAIChatCompletionClient

model_client = OpenAIChatCompletionClient(
    model="claude-sonnet-4.5",
    base_url="https://<your-gateway>/v1",
    api_key="vk_…",
    model_info=ModelInfo(
        vision=True, function_calling=True, json_output=True, structured_output=True, family="unknown",
    ),
)

async def main() -> None:
    agent = AssistantAgent("assistant", model_client=model_client)
    result = await agent.run(task="Say hello in Dutch.")
    print(result.messages[-1].content)
    await model_client.close()

asyncio.run(main())

For an OpenAI model name such as gpt-4o-mini you can leave model_info out. Set its fields to what the model really supports: AutoGen uses them to decide, for example, whether to send images or tools.

To use the Anthropic format instead:

from autogen_ext.models.anthropic import AnthropicChatCompletionClient

model_client = AnthropicChatCompletionClient(
    model="claude-sonnet-4.5",
    base_url="https://<your-gateway>",   # no /v1: the SDK adds /v1/messages
    api_key="vk_…",
    model_info=ModelInfo(
        vision=True, function_calling=True, json_output=True, structured_output=True, family="unknown",
    ),
)

The same Anthropic client can call gpt-4o-mini or gemini-3.5-flash: the gateway translates the request and the answer.

Agents in a team can share one client (all their calls are charged to one key) or each have their own: a cheaper model for the planner and a stronger one for the critic, for example. To tell agents apart in Request logs and in cost, give each its own key.

Streaming

Set model_client_stream=True on the agent and stream the run:

from autogen_agentchat.ui import Console

agent = AssistantAgent("assistant", model_client=model_client, model_client_stream=True)
await Console(agent.run_stream(task="Count to five."))

A stream from Claude or Gemini arrives in the format the client speaks (OpenAI chunks or Anthropic events). Vidai records the tokens and the cost either way.

Tool calls

Pass Python functions in tools; AutoGen describes them to the model and runs the calls. Every round trip is a call through the gateway, on any provider it translates to.

async def get_weather(city: str) -> str:
    """Weather for a city."""
    return f"Sunny in {city}"

agent = AssistantAgent("assistant", model_client=model_client, tools=[get_weather])
await Console(agent.run_stream(task="Weather in Utrecht?"))

The model must have function_calling=True in its model_info.

Reading what the gateway did

AutoGen doesn't hand you the response headers. Find the calls in Request logs: filter by Key or Model; each agent turn and tool round trip is its own call. The model that answered, any routing rule or fallback, and the cost are all there. On an error, the headers are on the exception (next section).

Errors

The gateway writes its errors in the format the client speaks, so the client raises the SDK's usual exception (openai.APIStatusError and its subclasses, or anthropic.APIStatusError). Match on the x-vidai-error-code header, not the message:

import openai

try:
    await agent.run(task="Hi")
except openai.APIStatusError as e:
    print(e.status_code, e.response.headers.get("x-vidai-error-code"))
    print(e.response.headers.get("x-vidai-request-id"))

Common codes: invalid_api_key (401), model_not_allowed, denied_by_routing_rule, content_guardrail_blocked (403), model_not_found (404), rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded (429), fallback_exhausted, upstream_timeout (5xx). The full list is in Headers and errors.

Model names

Send the names your organisation has approved, as listed under Models in the Control room (for example gpt-4o-mini, claude-sonnet-4.5, gemini-3.5-flash). GET /v1/models with your key returns exactly the models it may call. See Model names.

What goes through the gateway

Feature Through the gateway
Agents, teams, streaming, tools, structured output, vision (OpenAIChatCompletionClient) Yes, to OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models
The same with AnthropicChatCompletionClient Yes, to the same models, translated both ways
OpenAIAssistantAgent (Assistants API) No: see Assistants API

Check it works in a minute

python -c "import asyncio; from autogen_ext.models.openai import OpenAIChatCompletionClient as C; from autogen_core.models import UserMessage; print(asyncio.run(C(model='gpt-4o-mini', base_url='https://<your-gateway>/v1', api_key='vk_…').create([UserMessage(content='ping', source='user')])).content)"

A short answer means the address and key are right. The call shows in Request logs within seconds.

Good to know

  • With a model name starting claude- or gemini-, OpenAIChatCompletionClient sends calls straight to that provider (and reads ANTHROPIC_API_KEY or GEMINI_API_KEY) unless you pass base_url and api_key: always pass both.
  • The OpenAI client retries on its own (max_retries, two by default). A fallback chain may already have tried another model before you see an error; pass max_retries=0 if you don't want both.
  • The gateway's own 429 has no Retry-After header; back off on your side.
  • A team can run many turns; a max_turns or termination condition keeps one task from spending more than you meant.