Limits¶
Rate limits¶
Each key may have a limit in requests a minute. A key without its own limit follows your organisation's default for its kind of key (people's or agents' keys); with no default either, it has no limit. A key's page on Daily › Keys says which applies ("60 requests a minute", "600 requests a minute (the default)" or "No limit"), and shows a meter of this minute's calls.
| Situation | You get | What to do |
|---|---|---|
| The key's requests a minute are used up | 429 rate_limit_exceeded |
Wait, then retry with backoff: the minute starts again |
| The shared count can't be read, and the gateway is set to refuse rather than let calls through uncounted | 503 rate_limit_unavailable |
Retry shortly. It's an outage, not your limit |
| The provider's own limit is reached | The provider's 429, in your format, with its own headers | Follow the provider's retry-after |
- Limits are per key. Several keys don't share one limit, and a limit stays with the key when its secret is replaced.
- The gateway sends no rate-limit headers (
retry-after,x-ratelimit-*) on its own 429. Back off on your side: an exponential backoff starting at a second or two works well. - The count is shared between the gateway's instances every few seconds (every second unless your organisation changed it), so a key can briefly go over its limit by about that share of a minute's calls.
- Tokens aren't limited by the gateway: limit spend with a budget instead.
- A key's holder, or an agent's owner, may set a lower limit than the default. Only an admin over the key may set a higher one.
Request size¶
A request body over the gateway's size limit (100 MB unless whoever runs your gateway changed it) is refused with
413 payload_too_large, before anything is sent. A batch, fine-tuning or upload create body that names no model is
limited to 1 MiB. Send
large media by reference where the provider allows it (on Vertex AI, a Cloud Storage gs:// address), or through the
files endpoints.
Time limits¶
- A provider that doesn't answer in time gives
504 upstream_timeout. Retry, perhaps with a smaller request. - Trying the next model in a fallback chain is bounded: at most 3 attempts, 60 seconds and 1 MiB per call. A call larger than 1 MiB isn't sent again; you get the first model's error.
- Long streams: if anything of yours sits between your app and the gateway, give it long read time-outs.
Budgets¶
A budget your organisation sets can stop a key when it's reached. Your program then gets 403 api_key_disabled, with
error.reason such as budget_exceeded, until the budget's period starts again or someone raises it. The key's page
says which budget and until when.