Rate limits
Updated
Use the returned rate-limit and retry headers to pace work within your workspace's limits.
Stop if retries are refused; otherwise prefer Retry-After-Ms, then Retry-After, then SDK backoff.
See Billing and credits for charges and credit rules.
Limits
These defaults apply unless your workspace or key has a different limit.
| Limit | Default |
|---|---|
| Requests per minute (shared by all keys) | 600 |
| Calls in flight, per workspace | 16 |
| Decisions a UTC month, per workspace | 5,000,000 |
| Model calls a UTC day, per workspace | 100,000 |
| Requests a minute, per Playground key | 30 |
| Playground runs a day | 300 |
For workspaces using prepaid credits, the monthly default includes the billing decision ceiling. Your workspace can have a higher limit.
A decision is one answered call. A call that answers at least one question counts once toward the monthly cap, however many of its questions are refused. A call refused in full, a failed call and a replay of a stored answer do not count toward the monthly cap. Playground runs count. The month is the calendar month in UTC.
The daily cap counts every call sent to the model, including answered, refused, failed and Playground calls.
Replays count toward requests a minute.
| Code | Action |
|---|---|
rate_limit_exceeded | Wait for Retry-After and pace requests to the returned limits. |
concurrency_limit_exceeded | Reduce work in flight and follow Retry-After. |
workspace_limit_exceeded | Reduce concurrent work or input load and follow Retry-After. |
monthly_cap_reached | Follow resets_at and Retry-After for the workspace's monthly cap. |
daily_dispatch_limit | Wait until 00:00 UTC before sending more model calls. |
daily_unbilled_limit | Wait until 00:00 UTC before sending more work. |
playground_daily_limit | Wait for resets_at before another Playground run. |
capacity_exhausted | Wait for Retry-After and use bounded retries. |
These refusals return HTTP 429. The workspace's limit per minute is shared by all its keys.
An admission store timeout returns 503 service_unavailable. Wait for Retry-After and Retry-After-Ms before retrying.
Headers
The key window is checked first. A key refusal skips the workspace check. Otherwise, the reset uses the window with fewer requests remaining; ties select the key. Console-created keys have equal limits for both windows.
X-RateLimit-Limit- The lower key or workspace limit after both checks. A key refusal returns its own limit.
X-RateLimit-Remaining- The lower remaining count after both checks, or zero on a key refusal.
X-RateLimit-Reset- The Unix second when the window with fewer requests remaining resets; ties select the key. A key refusal returns its own reset.
Retry-After- The wait in seconds before retrying.
Retry-After-Ms- The wait in milliseconds; prefer it when both wait headers are present.
A rate_limit_exceeded refusal with error.details.cause set to replay_bytes means: Your account has used today's allowance for replayed answers. A request that repeats an Idempotency-Key gets its stored answer back, and those replays are limited by total bytes per account per UTC day. Replays resume at the next UTC day; the Retry-After headers give the wait.
Gateway 429 and 503 refusals carry Retry-After in whole seconds, rounded up, and Retry-After-Ms with the exact wait in milliseconds, except webhook_limit_exceeded.
Maintenance returns 503 with Retry-After and Retry-After-Ms. Each answer draws a wait of about 5 minutes (4 to 6), uniformly from 240 to 360 whole seconds.
Rate headers appear once the limiter has run. On a rate refusal, Retry-After uses the refusing window's reset. A monthly_cap_reached refusal includes monthly_request_cap, used and resets_at in error.details.
Retries
Read error.retryable and x-should-retry. Keep the same Idempotency-Key and JSON body for the decision's retries, and stop automatic retries when the flag is false.
See Idempotency for replay behavior and Errors for other refusals.