Audio¶
Speech to text and text to speech, through the gateway, with the code you already have: each call is checked, charged to the right team and recorded, and the recording itself is never kept.
What it is¶
OpenAI's three audio endpoints:
| Path | Does | SDK method |
|---|---|---|
POST /v1/audio/transcriptions |
Speech to text, in the spoken language | audio.transcriptions.create |
POST /v1/audio/translations |
Speech to English text | audio.translations.create |
POST /v1/audio/speech |
Text to speech | audio.speech.create |
The gateway passes them on unchanged to a provider that speaks the OpenAI API: OpenAI, Azure OpenAI, and providers added with the OpenAI kind of API that offer audio (Groq and Mistral, for example). They aren't translated: a call naming a Claude or Gemini model gets a 400.
The two settings¶
The same as for the OpenAI SDK: base address https://<your-gateway>/v1 and your Vidai key. See
OpenAI SDK for Python.
A minimal call¶
Transcription uploads the audio as a form:
from openai import OpenAI
client = OpenAI()
with open("meeting.mp3", "rb") as audio:
text = client.audio.transcriptions.create(model="whisper-1", file=audio)
print(text.text)
import fs from "node:fs";
import OpenAI from "openai";
const client = new OpenAI();
const text = await client.audio.transcriptions.create({
model: "whisper-1",
file: fs.createReadStream("meeting.mp3"),
});
console.log(text.text);
curl https://<your-gateway>/v1/audio/transcriptions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-F model=whisper-1 -F file=@meeting.mp3
Text to speech returns audio bytes:
with client.audio.speech.with_streaming_response.create(
model="gpt-4o-mini-tts", voice="alloy", input="Welcome to Fjordair.",
) as response:
response.stream_to_file("welcome.mp3")
const speech = await client.audio.speech.create({
model: "gpt-4o-mini-tts", voice: "alloy", input: "Welcome to Fjordair.",
});
fs.writeFileSync("welcome.mp3", Buffer.from(await speech.arrayBuffer()));
curl https://<your-gateway>/v1/audio/speech \
-H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
-d '{"model": "gpt-4o-mini-tts", "voice": "alloy", "input": "Welcome to Fjordair."}' \
-o welcome.mp3
The model names are examples: use one your organisation approved under Models.
Streaming¶
Speech audio streams to you as the provider sends it. A transcription model that streams its text (OpenAI's
stream=True on the newer transcription models) sends its events through unchanged.
Reading what the gateway did¶
Use with_raw_response (Python) or .withResponse() (TypeScript) and read x-vidai-request-id and x-vidai-model,
as shown on Chat Completions.
For an upload (transcriptions, translations), the request log keeps a summary: the size, the text fields such as
model and prompt, and the file's name and size. It never keeps the recording. The cost is in Request logs.
Errors¶
The codes are those of Chat Completions. Specific to these endpoints:
| Status | Code | When |
|---|---|---|
| 400 | bad_request |
The model is on a provider that doesn't speak the OpenAI API (Claude, Gemini, Bedrock) |
| 404 | endpoint_not_found |
The path is a near miss: /v1/audio/transcription (it's transcriptions), or a path that doesn't exist, such as /v1/audio/generations (text to speech is /v1/audio/speech) |
| 413 | payload_too_large |
The upload is over the gateway's size limit |
What goes through the gateway¶
| Feature | Through the gateway |
|---|---|
| Transcriptions, translations, speech | Yes, to OpenAI-format providers |
| Audio in chat messages | See Chat Completions: reaches models that take audio on the same provider |
| Realtime voice (WebSocket) | No, see Realtime |
Check it works in a minute¶
Run the speech curl above and play welcome.mp3. The call shows in Request logs within seconds.
Good to know¶
- Guardrails never scan the audio itself.