Skip to content

OCR, rerank and predictions

Three endpoints that aren't OpenAI's own but follow its style, offered by some providers. Through the gateway they're checked against your policies, charged to the right team and recorded like any call.

What it is

Path Does Offered by
POST /v1/ocr Reads text and layout from a document or image Mistral, for example (mistral-ocr-latest)
POST /v1/rerank Orders documents by relevance to a query Providers with a rerank API in this shape, such as Together AI
POST /v1/predictions, GET /v1/predictions/{id}, POST /v1/predictions/{id}/cancel Runs a model in the background and returns the result later Replicate

The gateway passes each one on unchanged to the provider that serves the model you name. It never translates them: the provider must offer the endpoint itself. The OpenAI SDK has no methods for these; call them over HTTP.

An admin adds Replicate under Providers › Add provider, and adds its models by hand: Replicate's catalogue is public, so Find models doesn't list them.

The two settings

The base address https://<your-gateway>/v1 and your Vidai key in Authorization: Bearer. See OpenAI SDK for Python.

A minimal call

curl https://<your-gateway>/v1/ocr \
  -H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "mistral-ocr-latest",
       "document": {"type": "document_url", "document_url": "https://example.com/invoice.pdf"}}'
curl https://<your-gateway>/v1/rerank \
  -H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "<a rerank model>", "query": "return policy",
       "documents": ["Orders ship in 2 days.", "Returns are free within 30 days."], "top_n": 1}'
curl https://<your-gateway>/v1/predictions \
  -H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "<a Replicate model>", "version": "<the model version, if it needs one>",
       "input": {"prompt": "A paper boat on a canal"}}'

For Replicate, model is the model's name in Vidai; version and input are Replicate's own. Create predictions at /v1/predictions (Replicate's per-model path, /v1/models/{owner}/{name}/predictions, isn't used through the gateway).

The model names must be ones your organisation approved under Models; the request body is the provider's own (see its API reference: Mistral OCR, Replicate predictions).

A prediction runs in the background. Its answer carries a link to check on it (urls.get); the gateway points that link back at itself, so poll the link exactly as it comes and the check reaches the right provider with no extra header.

Reading what the gateway did

Every answer carries x-vidai-request-id (paste it into Request logs › Request reference) and, when a model answered, x-vidai-model. The cost is in Request logs.

Errors

The codes are those of Chat Completions. Specific to these endpoints:

Status Code When
400 bad_request The model's provider doesn't speak this API
404 model_not_found No model of that name is set up
404 endpoint_not_found A near miss in the path

The provider's own error passes through when it doesn't offer the endpoint for that model.

What goes through the gateway

Feature Through the gateway
OCR, rerank, predictions (create, check, cancel) Yes, to the provider that offers them
Translation to another provider No

Check it works in a minute

Run the OCR or rerank curl above with a model your key may call (GET /v1/models lists them). An answer in the provider's own shape means the address, key and model are right; the call shows in Request logs within seconds.

Good to know

  • Rerank and predictions carry your documents and prompts out to the provider. Your own patterns apply to them. A VidaiGuard check set to mask or pseudonymise only records what it found here (the call goes through unchanged), and a pattern set to pseudonymise refuses the call: use mask instead if these calls should go through.
  • OCR sends a document link or file, which guardrails don't read.