Skip to content

Audio

Speech to text and text to speech, through the gateway, with the code you already have: each call is checked, charged to the right team and recorded, and the recording itself is never kept.

What it is

OpenAI's three audio endpoints:

Path Does SDK method
POST /v1/audio/transcriptions Speech to text, in the spoken language audio.transcriptions.create
POST /v1/audio/translations Speech to English text audio.translations.create
POST /v1/audio/speech Text to speech audio.speech.create

The gateway passes them on unchanged to a provider that speaks the OpenAI API: OpenAI, Azure OpenAI, and providers added with the OpenAI kind of API that offer audio (Groq and Mistral, for example). They aren't translated: a call naming a Claude or Gemini model gets a 400.

The two settings

The same as for the OpenAI SDK: base address https://<your-gateway>/v1 and your Vidai key. See OpenAI SDK for Python.

A minimal call

Transcription uploads the audio as a form:

from openai import OpenAI

client = OpenAI()

with open("meeting.mp3", "rb") as audio:
    text = client.audio.transcriptions.create(model="whisper-1", file=audio)
print(text.text)
import fs from "node:fs";
import OpenAI from "openai";

const client = new OpenAI();

const text = await client.audio.transcriptions.create({
  model: "whisper-1",
  file: fs.createReadStream("meeting.mp3"),
});
console.log(text.text);
curl https://<your-gateway>/v1/audio/transcriptions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F model=whisper-1 -F file=@meeting.mp3

Text to speech returns audio bytes:

with client.audio.speech.with_streaming_response.create(
    model="gpt-4o-mini-tts", voice="alloy", input="Welcome to Fjordair.",
) as response:
    response.stream_to_file("welcome.mp3")
const speech = await client.audio.speech.create({
  model: "gpt-4o-mini-tts", voice: "alloy", input: "Welcome to Fjordair.",
});
fs.writeFileSync("welcome.mp3", Buffer.from(await speech.arrayBuffer()));
curl https://<your-gateway>/v1/audio/speech \
  -H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o-mini-tts", "voice": "alloy", "input": "Welcome to Fjordair."}' \
  -o welcome.mp3

The model names are examples: use one your organisation approved under Models.

Streaming

Speech audio streams to you as the provider sends it. A transcription model that streams its text (OpenAI's stream=True on the newer transcription models) sends its events through unchanged.

Reading what the gateway did

Use with_raw_response (Python) or .withResponse() (TypeScript) and read x-vidai-request-id and x-vidai-model, as shown on Chat Completions.

For an upload (transcriptions, translations), the request log keeps a summary: the size, the text fields such as model and prompt, and the file's name and size. It never keeps the recording. The cost is in Request logs.

Errors

The codes are those of Chat Completions. Specific to these endpoints:

Status Code When
400 bad_request The model is on a provider that doesn't speak the OpenAI API (Claude, Gemini, Bedrock)
404 endpoint_not_found The path is a near miss: /v1/audio/transcription (it's transcriptions), or a path that doesn't exist, such as /v1/audio/generations (text to speech is /v1/audio/speech)
413 payload_too_large The upload is over the gateway's size limit

What goes through the gateway

Feature Through the gateway
Transcriptions, translations, speech Yes, to OpenAI-format providers
Audio in chat messages See Chat Completions: reaches models that take audio on the same provider
Realtime voice (WebSocket) No, see Realtime

Check it works in a minute

Run the speech curl above and play welcome.mp3. The call shows in Request logs within seconds.

Good to know

  • Guardrails never scan the audio itself.