Skip to content

Azure AI Foundry

Agents built in Microsoft Foundry (formerly Azure AI Foundry) can call their models through the gateway, as can the code you run there. Each call is then checked against your policies, charged to the right team and recorded in Request logs, beside the calls from everything else in your organisation.

What it is

Foundry is Microsoft's platform for building AI apps and agents. Two parts of it can go through the gateway:

  • Prompt agents in Foundry Agent Service, through a model connection to the gateway (Microsoft calls this bringing your own model through an AI gateway). The agent then uses a model behind the gateway instead of one deployed in Foundry.
  • Your own code running in or beside Foundry (a hosted agent, a function, a notebook), through an SDK pointed at the gateway.

Foundry's features that work only with models deployed in Foundry don't go through the gateway (see What goes through the gateway).

The two settings

For a prompt agent, add a model connection: in Foundry, open your project (or its parent resource), select Connections, then Add connection › Model, and choose the source for a non-Azure gateway:

Page Field Value
Source Connection name For example vidai
Base URL https://<your-gateway>/v1
Authentication API key Your Vidai key (starts vk_), never a provider's key
API key header name Authorization
API key header value Bearer {api_key}
Model configuration + Add model › Name A name your organisation has approved, for example gpt-4o-mini
Advanced Include deployment name in URL path Off
API version Empty

Leave Use the Responses API for upstream requests off, if it's offered: Foundry then calls https://<your-gateway>/v1/chat/completions, which reaches every provider your organisation has set up. Check that the Request URL Foundry shows ends in /v1/chat/completions before you choose Add.

For your own code, use the OpenAI SDK's two settings: OPENAI_BASE_URL = https://<your-gateway>/v1 and OPENAI_API_KEY = your Vidai key. Use the plain OpenAI client, not AzureOpenAI: the Azure client sends its key in an api-key header, which the gateway doesn't accept.

The gateway's address is under Connect your app on the Keys page and in System › Settings › Gateway. You get the key from Keys › Issue a key; its secret is shown once.

A minimal call

In a Foundry agent, the model is named <connection-name>/<model-name>:

  1. Create a prompt agent and choose the model vidai/gpt-4o-mini (your connection's name, then the model's).
  2. Send "Reply with exactly: ok" in the agent playground.

From code (Python, azure-ai-projects 2.0 or later):

import os
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition
from azure.identity import DefaultAzureCredential

project = AIProjectClient(endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], credential=DefaultAzureCredential())
agent = project.agents.create_version(
    agent_name="vidai-test",
    definition=PromptAgentDefinition(model="vidai/gpt-4o-mini", instructions="Answer briefly."),
)
openai = project.get_openai_client()
reply = openai.responses.create(
    input="Reply with exactly: ok",
    extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
)
print(reply.output_text)

Your code talks to Foundry; Foundry sends the model calls to the gateway over Chat Completions. Add claude-sonnet-4.5 or gemini-3.5-flash to the connection the same way: the gateway translates the request for that provider and the answer back into the OpenAI shape Foundry reads.

Tool calls

The agent's function tools are offered to the model in each call, and the model's tool calls come back through the gateway, on every provider it translates to. In Request logs, the Tool filter under More filters finds the calls that offered or called a tool by its name.

Reading what the gateway did

Foundry doesn't pass the gateway's response headers on to you. Find the agent's calls in Request logs: filter by Key (the key on the connection) or Model under More filters. Each call shows the model that answered, its cost, and which policy or rule acted on it.

Errors

A refused call fails the agent's run with the gateway's message. The common ones:

Status Gateway code What to do
401 invalid_api_key, missing_api_key Check the connection's key, header name and header value (Bearer {api_key})
403 model_not_allowed, denied_by_routing_rule Use a model the key may call
403 content_guardrail_blocked A guardrail stopped the content
404 model_not_found Check the model's Name on the connection against Models
429 rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded Slow down

If Foundry itself says model not found, the agent's model isn't written as <connection-name>/<model-name>.

Model names

On the connection, add each model by the name your organisation has approved, as listed under Models in the Control room. A GET to https://<your-gateway>/v1/models with the key lists exactly the models it may call.

What goes through the gateway

In Foundry Through the gateway
Prompt agents whose model comes from the gateway connection Yes, over Chat Completions, to every provider your organisation has set up
The same, with Use the Responses API for upstream requests on To OpenAI and Azure OpenAI models only; another provider gets a clear 400
Your own code with the OpenAI, Anthropic or Google GenAI SDK pointed at the gateway Yes
Agents and apps using models deployed in Foundry, Foundry IQ, agent memory, agent optimisation, real-time audio No: Microsoft lists these as not working with connected models

Check it works in a minute

Send "Reply with exactly: ok" to the agent in the playground. ok means the connection works. The call shows in Request logs within seconds.

Good to know

  • Foundry calls the gateway from Microsoft's network: the gateway must be reachable from it, over the internet or through the private networking Microsoft describes for connected models.
  • One connection, one key: give each Foundry project its own connection and key (an agent's key works well) so each project's spend is apart.
  • Foundry's own content safety doesn't run on connected models; your guardrails in Vidai do, on every call.
  • Bringing your own model to Foundry Agent Service is described in Microsoft's Bring your own model page.