SE Home
Help
  • Documentation
  • Help centre
SEScrape Engine

Getting started

DocumentationQuickstartAuthenticationPlayground

Extracting data

Selecting valuesTransformsLists & tablesPage dataAI extractionMCP server

Fetching pages

Custom requestsTimeouts & redirectsCachingAutomatic retriesFallback strategies

Browsers & actions

BrowsersActionsRecipesScreenshots & PDFStored files

Getting past blocks

Anti-bot protectionCaptchasProxiesFingerprintsSessions

Workflows

Multi-step requestsVariablesPaginationBatchCrawlingTemplatesSchedules & monitoringAsync jobs & webhooksExport to S3, Sheets, databases & webhooks

Account & billing

CreditsSecretsPlansLimitsLogs & request idsNotificationsAccountAffiliate programmeHelp centre

Reference

POST /scrapeGET /sessions/:idGET /browsersGET /proxies/countriesGET /healthErrors
Documentation

Getting started

DocumentationQuickstartAuthenticationPlayground

Extracting data

Selecting valuesTransformsLists & tablesPage dataAI extractionMCP server

Fetching pages

Custom requestsTimeouts & redirectsCachingAutomatic retriesFallback strategies

Browsers & actions

BrowsersActionsRecipesScreenshots & PDFStored files

Getting past blocks

Anti-bot protectionCaptchasProxiesFingerprintsSessions

Workflows

Multi-step requestsVariablesPaginationBatchCrawlingTemplatesSchedules & monitoringAsync jobs & webhooksExport to S3, Sheets, databases & webhooks

Account & billing

CreditsSecretsPlansLimitsLogs & request idsNotificationsAccountAffiliate programmeHelp centre

Reference

POST /scrapeGET /sessions/:idGET /browsersGET /proxies/countriesGET /healthErrors

Caching

The cache is shared by every user of this API. Don't rely on it for pages that depend on who is asking, unless the request carries what identifies them (a cookie or a header), which gives it its own copy.

Scraping the same page again and again is slow and can get you blocked. With options.cache_ttl, the API answers from a copy of the page it fetched recently, without contacting the site.

"options": { "cache_ttl": 300 }

cache_ttl is the maximum age, in seconds, of a copy you accept, from 0 to 86400 (24 hours). 0, the default, always fetches the page.

See it work

The demo page shows the time it was generated. Send the request twice: the second answer comes from the cache, with the same time, "cached": true and the copy's age in cache_age (both in meta):

Actions

Data

{
  "status": 200,
  "data": ["2026-10-18T14:03:12.511Z"],
  "meta": { "cached": true, "cache_age": 4, "attempts": 0, … }
}

How it works

  • The API keeps the page, not the extracted values: a cached page answers any fields.
  • Every page fetched is kept, even without cache_ttl. Each request decides how old a copy it accepts: one request can ask for a page fetched in the last 10 seconds, another accepts one up to an hour old.
  • Copies are kept 24 hours at most.
  • A cached answer has "attempts": 0 in its meta: the site wasn't contacted.

What makes two requests share a copy

Two requests share a copy only when the site would answer them the same way:

Shares the copyGets its own copy
Different fieldsDifferent target_url
Different retry, backoff, timeoutDifferent method, headers, cookies or body
Different follow_redirects, fallback, proxy location filters or list, or fingerprint

Errors are never cached: a failed request is tried again next time.

To keep a page out of the cache entirely, set "cache_ttl": false: it's neither read from nor stored in the cache.

On this page

  • See it work
  • How it works
  • What makes two requests share a copy