Authentication

Base URLCopy Anchor Link

Every endpoint lives under /v1 of the API. In these pages, the examples read it from an environment variable:

export SCRAPE_ENGINE_URL="https://<your API host>/v1"
export SCRAPE_ENGINE_KEY="sk_…"

API keysCopy Anchor Link

Create a key in Dashboard → API keys. It's shown once: copy it into your secrets manager or environment. We only keep a hash, so a lost key can't be recovered; revoke it and create a new one.

Send it with each request, in either header:

curl "$SCRAPE_ENGINE_URL/scrape" \
  -H "Authorization: Bearer $SCRAPE_ENGINE_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "target_url": "https://example.com", "fields": ["h1"] }'

curl "$SCRAPE_ENGINE_URL/browsers" -H "x-api-key: $SCRAPE_ENGINE_KEY"

Each key allows 60 requests per minute. Use one key per app or server, so you can revoke one without touching the others.

Expiry and rotationCopy Anchor Link

A key can expire on its own: pick 30, 90 or 365 days when you create it (or never). The keys page shows when each one expires, in red in its last week; after that date, requests with it answer 401 UNAUTHORIZED.

Rotate a key (the ⟳ button) to replace it without downtime: you get a new key with the same name and lifetime, and the old one keeps working for 24 hours, time to deploy the new one everywhere. Rotate when someone who knew it leaves, when it may have leaked, or on a schedule.

What we storeCopy Anchor Link

Logs, results, captures and monitoring snapshots are encrypted at rest. Credentials in a saved template or schedule (Authorization or Cookie headers, cookies, a password typed by an action) must be secrets: {{secrets.NAME}}. Written in the clear, the template or schedule is refused.

Credits in responsesCopy Anchor Link

Every response says what it cost, in headers:

See Credits for how a request is priced.

ErrorsCopy Anchor Link

Errors of the scrape itself (timeouts, blocks, bad selectors) are listed in Errors.