Skip to content

CrewAI

Your crews, agents and tasks stay as they are. Give the LLM object three settings and every call the crew makes, delegations and tool round trips included, is checked against your policies, charged to the right team and recorded in Request logs. One setting covers every provider, because the gateway translates the OpenAI format to Claude and Gemini.

What it is

The crewai package for Python. Agents call models through its LLM class. CrewAI has its own client for OpenAI's format, which reaches the gateway at /v1; with custom_openai=True it sends any model name there, whatever the provider behind it.

The three settings

Setting Environment variable LLM(...) argument Value
Base address OPENAI_BASE_URL (OPENAI_API_BASE is also read) base_url https://<your-gateway>/v1
Key OPENAI_API_KEY api_key Your Vidai key (starts vk_), never a provider's key
Use the OpenAI format for any model custom_openai=True Needed for model names CrewAI doesn't know as OpenAI's

The key is sent as Authorization: Bearer. You get it from Keys › Issue a key, or, for an agent, from the agent's Keys section under Applications & agents; its secret is shown once. The gateway's address and the base URL for each kind of SDK are under Connect your app on the Keys page and in System › Settings › Gateway.

export OPENAI_BASE_URL="https://<your-gateway>/v1"
export OPENAI_API_KEY="vk_…"

A minimal call

from crewai import Agent, Crew, LLM, Task

llm = LLM(
    model="claude-sonnet-4.5",  # or "gpt-4o-mini", "gemini-3.5-flash"
    custom_openai=True,         # send it in the OpenAI format to the gateway
    base_url="https://<your-gateway>/v1",
    api_key="vk_…",
)

writer = Agent(role="Writer", goal="Write short greetings", backstory="Brief and friendly.", llm=llm)
task = Task(description="Say hello in Dutch.", expected_output="One line.", agent=writer)
print(Crew(agents=[writer], tasks=[task]).kickoff())

The model name is the one your organisation approved, as is, with or without an openai/ prefix (CrewAI removes it). Without custom_openai=True, CrewAI looks the name up in its own list of each provider's models and, for a name it doesn't know (an alias, for example), hands it to another library instead of its OpenAI client. custom_openai=True keeps every call on the gateway in one format.

Give each agent its own LLM to use different models, or different keys, in one crew.

Streaming

llm = LLM(model="gpt-4o-mini", custom_openai=True, base_url="https://<your-gateway>/v1", api_key="vk_…", stream=True)

A stream from Claude or Gemini arrives as OpenAI chunks. Vidai records the tokens and the cost either way.

Tool calls

CrewAI tools (the @tool decorator or BaseTool) are sent as OpenAI tools, and work on every provider the gateway translates to. Structured output (output_pydantic= or output_json= on a task) works the same way. Each step of an agent is its own call, so one task shows as several calls in Request logs (filter Tool to see the ones that offered or called a tool).

Reading what the gateway did

CrewAI doesn't pass the response headers on. Look the calls up in Request logs: filter Key (or Agent, when the key is an agent's), Model or Path. Each call shows the model that answered, what translation or a guardrail changed, the routing rule or fallback that acted, and what it cost. To read the headers themselves, make the same call once with the OpenAI SDK for Python.

Errors

The gateway writes its errors in the OpenAI format, so CrewAI raises the openai package's exceptions (or reports them in the agent's output, depending on where they happen). The status says what kind of error it is; the call's row in Request logs gives the gateway's reason (filter Exact status or Why it failed).

Status Common gateway codes
401 missing_api_key, invalid_api_key, conflicting_auth_headers
403 api_key_disabled, model_not_allowed, denied_by_routing_rule, content_guardrail_blocked
404 model_not_found, endpoint_not_found
429 rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded
5xx ai_traffic_suspended, fallback_exhausted, upstream_timeout, upstream_unreachable

A key that's switched off carries a short reason beside api_key_disabled (for example agent_suspended); treat the list of reasons as open-ended.

Model names

Send the names your organisation approved, as listed under Models in the Control room. GET /v1/models on the gateway returns exactly the models your key may call. An alias set by an admin works like any other name.

What goes through the gateway

Feature Through the gateway
Crews, agents, tasks, delegation, tools, structured output, streaming (Chat Completions) Yes, to OpenAI, Azure OpenAI, Anthropic, Gemini and Vertex AI models
LLM(api="responses") Yes, to OpenAI and Azure OpenAI only; another provider gets a clear 400
Memory and knowledge embeddings Only with an OpenAI-format embedder pointed at the gateway, to OpenAI and Azure OpenAI models

The embedder takes the gateway's address as api_base (it doesn't read OPENAI_BASE_URL):

embedder = {"provider": "openai", "config": {
    "api_base": "https://<your-gateway>/v1", "api_key": "vk_…", "model_name": "text-embedding-3-small"}}

Check it works in a minute

python -c "from crewai import LLM; print(LLM(model='gpt-4o-mini', custom_openai=True).call('Reply with ok.'))"

With the two environment variables set, a short answer means the address and key are right. The call shows in Request logs within seconds.

Good to know

  • CrewAI's OpenAI client retries twice by default. A fallback chain may already have tried another model first; set max_retries=0 on LLM if you don't want both.
  • The gateway's own 429 has no Retry-After header; back off on your side.
  • Keep api at its default ("completions") for models that aren't OpenAI's: the Responses API reaches only OpenAI and Azure OpenAI through the gateway.