OCR, rerank and predictions¶
Three endpoints that aren't OpenAI's own but follow its style, offered by some providers. Through the gateway they're checked against your policies, charged to the right team and recorded like any call.
What it is¶
| Path | Does | Offered by |
|---|---|---|
POST /v1/ocr |
Reads text and layout from a document or image | Mistral, for example (mistral-ocr-latest) |
POST /v1/rerank |
Orders documents by relevance to a query | Providers with a rerank API in this shape, such as Together AI |
POST /v1/predictions, GET /v1/predictions/{id}, POST /v1/predictions/{id}/cancel |
Runs a model in the background and returns the result later | Replicate |
The gateway passes each one on unchanged to the provider that serves the model you name. It never translates them: the provider must offer the endpoint itself. The OpenAI SDK has no methods for these; call them over HTTP.
An admin adds Replicate under Providers › Add provider, and adds its models by hand: Replicate's catalogue is public, so Find models doesn't list them.
The two settings¶
The base address https://<your-gateway>/v1 and your Vidai key in Authorization: Bearer. See
OpenAI SDK for Python.
A minimal call¶
curl https://<your-gateway>/v1/ocr \
-H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
-d '{"model": "mistral-ocr-latest",
"document": {"type": "document_url", "document_url": "https://example.com/invoice.pdf"}}'
curl https://<your-gateway>/v1/rerank \
-H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
-d '{"model": "<a rerank model>", "query": "return policy",
"documents": ["Orders ship in 2 days.", "Returns are free within 30 days."], "top_n": 1}'
curl https://<your-gateway>/v1/predictions \
-H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
-d '{"model": "<a Replicate model>", "version": "<the model version, if it needs one>",
"input": {"prompt": "A paper boat on a canal"}}'
For Replicate, model is the model's name in Vidai; version and input are Replicate's own. Create predictions at
/v1/predictions (Replicate's per-model path, /v1/models/{owner}/{name}/predictions, isn't used through the gateway).
The model names must be ones your organisation approved under Models; the request body is the provider's own (see its API reference: Mistral OCR, Replicate predictions).
A prediction runs in the background. Its answer carries a link to check on it (urls.get); the gateway points that
link back at itself, so poll the link exactly as it comes and the check reaches the right provider with no extra
header.
Reading what the gateway did¶
Every answer carries x-vidai-request-id (paste it into Request logs › Request reference) and, when a model
answered, x-vidai-model. The cost is in Request logs.
Errors¶
The codes are those of Chat Completions. Specific to these endpoints:
| Status | Code | When |
|---|---|---|
| 400 | bad_request |
The model's provider doesn't speak this API |
| 404 | model_not_found |
No model of that name is set up |
| 404 | endpoint_not_found |
A near miss in the path |
The provider's own error passes through when it doesn't offer the endpoint for that model.
What goes through the gateway¶
| Feature | Through the gateway |
|---|---|
| OCR, rerank, predictions (create, check, cancel) | Yes, to the provider that offers them |
| Translation to another provider | No |
Check it works in a minute¶
Run the OCR or rerank curl above with a model your key may call (GET /v1/models lists them). An answer in the
provider's own shape means the address, key and model are right; the call shows in Request logs within seconds.
Good to know¶
- Rerank and predictions carry your documents and prompts out to the provider. Your own patterns apply to them. A VidaiGuard check set to mask or pseudonymise only records what it found here (the call goes through unchanged), and a pattern set to pseudonymise refuses the call: use mask instead if these calls should go through.
- OCR sends a document link or file, which guardrails don't read.