Skip to content

The parts: the gateway and the Control room

Before a security review or an architecture board, people ask the same questions: what runs where, what talks to what, where do prompts go, and how fast does a change take effect? This page answers them once.

Vidai has two parts you meet, and both run in your own infrastructure:

  • The gateway carries every AI call. Your programs call it instead of calling a provider.
  • The Control room is where people set the rules and read what happened: the screens, and the service behind them that keeps your organisation's people, keys, rules, prices and records.

They keep their records in a database you run. VidaiGuard Enterprise, Vidai's own models that check content, runs beside the gateway once it's set up; until then, guardrails check content with patterns alone.

flowchart LR
  apps["Your programs, agents and tools<br/>(their own SDK, a Vidai key)"] -- "AI calls" --> gw{{"Gateway<br/>one or more nodes"}}
  gw -- "calls, with your provider keys" --> prov["Providers<br/>cloud, or your own servers"]
  gw <-. "content checks" .-> guard["VidaiGuard<br/>(beside the gateway)"]
  people["People in a browser"] --> cr["Control room"]
  scripts["Your scripts and CI<br/>(API tokens)"] --> cr
  cr -- "rules and keys,<br/>within seconds" --> gw
  gw -- "what each call did" --> db[("Database")]
  cr <--> db
  subgraph yours["Your infrastructure"]
    gw
    guard
    cr
    db
  end

What each part does

Part What it does Who talks to it
Gateway Checks each call's key and every rule that reaches it, sends the call to a provider with your provider keys, translates between SDK formats, and records what happened. A single small binary; several nodes can run side by side Your programs, with a Vidai key
Control room Keeps people, teams, applications, agents, keys, providers, models, prices, rules and settings; tells the gateway what to allow; prices every call and lists it; produces evidence People in a browser; your own systems through the management API, with an API token
Database Holds the organisation, the rules and the record of every call The gateway and the Control room
VidaiGuard Enterprise Finds what a pattern can't (a name in free text, a disguised injection attempt) for guardrails The gateway

A program never talks to the Control room to make an AI call, and a person never talks to the gateway: keys are for calls, API tokens and sign-in are for managing Vidai. See Keys and Service accounts and API tokens.

What Enterprise includes depends on the features your licence includes; see Editions and licences.

Where your data lives

  • Prompts and answers pass through the gateway on their way to the provider you chose, and nowhere else. Whether the gateway keeps a call's full content depends on how its request logging is set; reading kept content is a role of its own, the Content reader, and every read is recorded. See Request logs and the call record.
  • Provider keys are held by Vidai and used by the gateway. They never sit in your apps, and a provider key is shown by its label, never its secret.
  • Content checks run in the gateway (patterns) and, where it's set up, on VidaiGuard beside it, so the data being checked stays in your infrastructure.
  • The licence is checked where Vidai runs, offline. Vidai doesn't phone home.

How fast a change takes effect

  • A change in the Control room (a key turned off, a model withdrawn, a rule enforced) reaches the gateway within seconds.
  • A call through the gateway shows in Request logs within seconds. Cost totals, the dashboards and a key's Last used count calls up to about two minutes old.
  • With several gateway nodes, System › Status lists each instance and whether it's Up to date or Picking up the latest change. Stop AI traffic is read back from every node, and Not taken everywhere shows if one disagrees: a partial stop never reads as a full one. See Safety switches.

System › Status: Everything is working, every check, the gateway and each of its instances up to date

How fast the gateway is

The gateway adds well under a millisecond to a call. In the gateway's 0.9.0-beta.5 run it handled 47,200 calls a second at 0.58 ms median: one machine, 32 concurrent callers, every call recorded, a mock provider in place of a real one. It's our own measurement, not a published benchmark, and it's updated each release.