Endpoints

Map

Every URL on a site in one call, optionally ranked against a query.

POST/v1/map

Map discovers the URLs on a site (up to 100,000) by combining sitemaps, robots.txt, the link graph and a render fallback for JavaScript-only navigation. Pass search to rank the results by relevance, or path globs to filter them. 2 credits per 10 fetches the map makes (robots.txt, sitemap files, pages fetched for links, and the render fallback each count as one), rounded up, with a minimum of 2, however many URLs come back. A small site usually costs 2. One call costs at most 28: that much is reserved up front, then settled to the actual count, which credits_used reports.

If you have fewer than 28 credits left (or a playground key has less than that left of its budget), the map still runs: it reserves the largest even amount you can afford, at least 2, and stops fetching once that is used up, so it never costs more than was reserved. Every fetch counts against that budget, robots.txt included. The start page and sitemaps come first, then the link walk. A map cut short this way returns what it found with "budgetLimited": true. Only when even 2 credits aren't available does it fail with insufficient_credits.

Example#

curl -X POST "https://api.vermin.dev/v1/map" \
  -H "Authorization: Bearer $VERMIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://docs.example.com",
    "search": "authentication",
    "limit": 100
  }'

Request#

MapRequest

urlstring (uri)required
searchstring
Rank URLs by relevance to this query
limitintegerdefault 50001–100000
includeSubdomainsbooleandefault false
sitemap"include" | "skip" | "only"default "include"
includePathsstring[]
Only return URLs whose path matches one of these globs (CrawlRequest.includePaths syntax).
excludePathsstring[]
Never return URLs whose path matches one of these globs.

Response#

200 OK

{
  "success": true,
  "links": [
    { "url": "https://docs.example.com/guides/auth", "title": "Authentication", "source": "sitemap" },
    { "url": "https://docs.example.com/api/keys", "title": "API keys", "source": "link" },
    { "url": "https://docs.example.com/app#settings", "source": "render" }
  ],
  "credits_used": 2,
  "request_id": "req_01J9V5B2QX"
}

budgetLimited is only present (as true) on a map that stopped early for lack of credits.

source tells you where each URL came from: sitemap, link (found by following links) or render (only visible after running the page's JavaScript).