Azure AI Foundry¶
Agents built in Microsoft Foundry (formerly Azure AI Foundry) can call their models through the gateway, as can the code you run there. Each call is then checked against your policies, charged to the right team and recorded in Request logs, beside the calls from everything else in your organisation.
What it is¶
Foundry is Microsoft's platform for building AI apps and agents. Two parts of it can go through the gateway:
- Prompt agents in Foundry Agent Service, through a model connection to the gateway (Microsoft calls this bringing your own model through an AI gateway). The agent then uses a model behind the gateway instead of one deployed in Foundry.
- Your own code running in or beside Foundry (a hosted agent, a function, a notebook), through an SDK pointed at the gateway.
Foundry's features that work only with models deployed in Foundry don't go through the gateway (see What goes through the gateway).
The two settings¶
For a prompt agent, add a model connection: in Foundry, open your project (or its parent resource), select Connections, then Add connection › Model, and choose the source for a non-Azure gateway:
| Page | Field | Value |
|---|---|---|
| Source | Connection name | For example vidai |
| Base URL | https://<your-gateway>/v1 |
|
| Authentication | API key | Your Vidai key (starts vk_), never a provider's key |
| API key header name | Authorization |
|
| API key header value | Bearer {api_key} |
|
| Model configuration | + Add model › Name | A name your organisation has approved, for example gpt-4o-mini |
| Advanced | Include deployment name in URL path | Off |
| API version | Empty |
Leave Use the Responses API for upstream requests off, if it's offered: Foundry then calls
https://<your-gateway>/v1/chat/completions, which reaches every provider your organisation has set up. Check that the
Request URL Foundry shows ends in /v1/chat/completions before you choose Add.
For your own code, use the OpenAI SDK's two settings: OPENAI_BASE_URL = https://<your-gateway>/v1 and
OPENAI_API_KEY = your Vidai key. Use the plain OpenAI client, not AzureOpenAI: the Azure client sends its key in
an api-key header, which the gateway doesn't accept.
The gateway's address is under Connect your app on the Keys page and in System › Settings › Gateway. You get the key from Keys › Issue a key; its secret is shown once.
A minimal call¶
In a Foundry agent, the model is named <connection-name>/<model-name>:
- Create a prompt agent and choose the model
vidai/gpt-4o-mini(your connection's name, then the model's). - Send "Reply with exactly: ok" in the agent playground.
From code (Python, azure-ai-projects 2.0 or later):
import os
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition
from azure.identity import DefaultAzureCredential
project = AIProjectClient(endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], credential=DefaultAzureCredential())
agent = project.agents.create_version(
agent_name="vidai-test",
definition=PromptAgentDefinition(model="vidai/gpt-4o-mini", instructions="Answer briefly."),
)
openai = project.get_openai_client()
reply = openai.responses.create(
input="Reply with exactly: ok",
extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
)
print(reply.output_text)
Your code talks to Foundry; Foundry sends the model calls to the gateway over Chat Completions. Add claude-sonnet-4.5
or gemini-3.5-flash to the connection the same way: the gateway translates the request for that provider and the
answer back into the OpenAI shape Foundry reads.
Tool calls¶
The agent's function tools are offered to the model in each call, and the model's tool calls come back through the gateway, on every provider it translates to. In Request logs, the Tool filter under More filters finds the calls that offered or called a tool by its name.
Reading what the gateway did¶
Foundry doesn't pass the gateway's response headers on to you. Find the agent's calls in Request logs: filter by Key (the key on the connection) or Model under More filters. Each call shows the model that answered, its cost, and which policy or rule acted on it.
Errors¶
A refused call fails the agent's run with the gateway's message. The common ones:
| Status | Gateway code | What to do |
|---|---|---|
| 401 | invalid_api_key, missing_api_key |
Check the connection's key, header name and header value (Bearer {api_key}) |
| 403 | model_not_allowed, denied_by_routing_rule |
Use a model the key may call |
| 403 | content_guardrail_blocked |
A guardrail stopped the content |
| 404 | model_not_found |
Check the model's Name on the connection against Models |
| 429 | rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded |
Slow down |
If Foundry itself says model not found, the agent's model isn't written as <connection-name>/<model-name>.
Model names¶
On the connection, add each model by the name your organisation has approved, as listed under Models in the Control
room. A GET to https://<your-gateway>/v1/models with the key lists exactly the models it may call.
What goes through the gateway¶
| In Foundry | Through the gateway |
|---|---|
| Prompt agents whose model comes from the gateway connection | Yes, over Chat Completions, to every provider your organisation has set up |
| The same, with Use the Responses API for upstream requests on | To OpenAI and Azure OpenAI models only; another provider gets a clear 400 |
| Your own code with the OpenAI, Anthropic or Google GenAI SDK pointed at the gateway | Yes |
| Agents and apps using models deployed in Foundry, Foundry IQ, agent memory, agent optimisation, real-time audio | No: Microsoft lists these as not working with connected models |
Check it works in a minute¶
Send "Reply with exactly: ok" to the agent in the playground. ok means the connection works. The call shows in
Request logs within seconds.
Good to know¶
- Foundry calls the gateway from Microsoft's network: the gateway must be reachable from it, over the internet or through the private networking Microsoft describes for connected models.
- One connection, one key: give each Foundry project its own connection and key (an agent's key works well) so each project's spend is apart.
- Foundry's own content safety doesn't run on connected models; your guardrails in Vidai do, on every call.
- Bringing your own model to Foundry Agent Service is described in Microsoft's Bring your own model page.