Skip to content

The stack nobody chose

If you run a platform team, you have probably thought it: we could build this. Put a small service in front of the model providers, hand out keys, log the calls. A homemade gateway is a weekend.

What grows around it isn't. This page walks through what the weekend project turns into once real teams, real invoices and a real auditor arrive, piece by piece. Nobody chooses this stack. It accumulates, one urgent request at a time.

Week one: the calls

The first service forwards OpenAI-shaped requests. Then a team arrives with the Anthropic SDK, another with Gemini. Now you need translation across three SDK dialects: requests, answers, streams event by event, tool calls and their results, reasoning and structured output, in both directions. Every provider release changes a field.

Then a provider has a bad afternoon, so you add failover. It has to happen before the first byte reaches the caller, never mid-stream; it mustn't send a large upload three times; it mustn't charge you twice without telling you; and it has to say afterwards which models it tried.

Vidai does this in the gateway, for every app: translation between the OpenAI, Anthropic and Gemini formats, and fallback chains that answer before anything reaches the caller and record every model they tried.

Month one: the money

Finance asks what AI cost last month, by team. Now you need per-call pricing: a price for every model, input and output and cached tokens priced apart, price history so a call made in March is priced at March's price, several currencies that are never silently added together, and calls with no price that are counted as unknown, not as zero. When a price arrives late you need to price past calls again, with a dry run first, without touching a month Finance has already closed.

Then a budget, which has to warn, stop or switch to a cheaper model, at any level from one key to the whole organisation.

In Vidai, every call is priced within minutes at the price in force when it was made, from the rate card, and counts against budgets that warn, stop or reroute. See Cost and Prices and the rate card.

Cost › Calls: one priced call with its cost, who paid, its tokens, the price in force and the markup

Quarter one: the rules

Security wants personal data kept out of prompts. Legal wants calls kept in the EU. A department wants its own models. Now you need policy that composes across levels: set once for the organisation, tightened by a team, never loosened below; and a way to answer "why was this call refused?" by showing which rule, set where, decided it.

Pattern matching catches card numbers and email addresses. It doesn't catch a name in free text or a disguised injection attempt, so you need models that find personal data, run somewhere your data may go.

Vidai sets rules at levels, shows What applies here for any key, team or the organisation, and checks content with Guardrails: pattern checks, and VidaiGuard checks whose models run beside your gateway.

The first audit

The auditor asks for proof. Screenshots won't do. Now you need sealed, verifiable evidence: records chained so a change or a gap shows, signed on both sides, a report the auditor can check without access to your systems, and a sample method they can reproduce. You need retention set to what each regulation asks, legal hold, and answers to data subject requests.

Vidai keeps a tamper-evident audit log, seals call records on both sides with tamper evidence, and produces an auditor report an auditor checks alone, on their own machine.

Everything around it

Then come the parts every internal tool needs and nobody budgets for:

  • People and access: single sign-on, roles at levels, approvals by a second admin, access that ends on a date, and an emergency admin so nobody is ever locked out.
  • Dashboards per role: an admin, a FinOps person and a compliance officer each ask a different question.
  • Audit: who changed what, from where and why, people and scripts alike.
  • Incidents: one switch that stops every AI call on every gateway node, and proof that every node took it.

One library per concern

Each of these is a library, a script or a service of its own, each with its upgrades, its on-call rota and its own idea of what a "team" is. The weekend gateway is now a platform, and your best engineers run it instead of building what your company sells.

What your team does instead

With Vidai, the gateway carries every call and the Control room holds the rules, the cost and the evidence. Your programs change one line: the base address and the key. Your platform team spends its time on your products, and answers the questions above from one screen:

Request logs: a call sent to an EU deployment by a compliance route, with Why this happened naming the route and where it was set

Start with How Vidai works, or see a day with Vidai.