Rate limits

Limits are per organisation and depend on your plan. They exist so one runaway script can't starve everybody else's rats.

• Requests per minute counts every API call, including polling GET /v1/crawl/{id}.
• Concurrent requests is how many requests may be in flight at once (browser-heavy scrapes hold a slot until they finish).
• Concurrent crawls is how many crawl jobs may be queued or running at once. Extra crawls are rejected, not queued.

Rate-limit headers#

Every authenticated /v1 response carries the IETF RateLimit header fields, so clients and agents can throttle themselves before they hit a 429:

The bucket refills continuously (a token bucket), so r climbs back one request at a time rather than resetting at the top of the minute.

When you hit a limit#

You get a 429 rate_limited with retry_after_ms in the body and a Retry-After header (seconds). It costs nothing.

{
  "success": false,
  "error": { "code": "rate_limited", "message": "120 requests/min exceeded", "retry_after_ms": 850 },
  "credits_used": 0,
  "request_id": "req_01J9V3M2A1"
}

Both SDKs retry 429, 503 capacity_exceeded and transient network errors automatically with exponential backoff and jitter, honouring retry_after_ms. If you roll your own client, do the same: wait at least retry_after_ms, then double the delay on each further failure.

Need more?#

Higher plans raise every limit, and Enterprise limits are set per contract. Talk to us if you need a burst for a backfill.