API guide

A short guide to authenticating, starting an Actor, waiting for it to finish, and reading the results. For request and response details, see the API overview and the reference pages linked at the end. If you'd rather not write raw HTTP calls, use the JavaScript client or the Python client.

Base URL: https://api.scrape.ly
Public endpoints live under /v2.

Authenticate

Create a token in the dashboard under Settings → API & Integrations. Send it as a Bearer token:

export SCRAPELY_TOKEN="scrapely_..."

curl -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  https://api.scrape.ly/v2/users/me

The API only reads the token from the Authorization header. A ?token= query parameter is ignored. Treat tokens like passwords.

Start a run

POST /v2/actors/{actorId}/runs starts a run and returns right away. {actorId} is the Actor ID or the named ID username~actor-name.

The JSON body is the Actor input object at the top level. Do not wrap it in { "input": ... } or { "runInput": ... }.

curl -X POST https://api.scrape.ly/v2/actors/janedoe~my-actor/runs \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "maxResults": 50}'

Response (201, or 202 if the run is queued):

{
  "data": {
    "id": "a7XyZ8kLmN2pQ4rSt",
    "actId": "eLN5rTB8sYTBWdfzx",
    "status": "READY",
    "defaultDatasetId": "dS9t8uVwXyZ12345",
    "defaultKeyValueStoreId": "kVxYzAbCdEf12345",
    "defaultRequestQueueId": "rQmNoPqRsT123456"
  }
}

Optional query parameters: build, memory, timeout, maxItems, maxTotalChargeUsd, restartOnError, waitForFinish (up to 60; the wait stops at 45 seconds), webhooks, forcePermissionLevel. build is owner only: on someone else's Actor it's ignored and the run uses the User Build. See Start a run.

Poll until it finishes

For longer jobs, poll GET /v2/actor-runs/{runId} until status is terminal:

SUCCEEDED, FAILED, ABORTED, or TIMED-OUT.

RUN_ID=a7XyZ8kLmN2pQ4rSt

curl -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  https://api.scrape.ly/v2/actor-runs/$RUN_ID

Prefer webhooks over tight polling. waitForFinish on POST .../runs and on GET /v2/actor-runs/{runId} blocks for at most 45 seconds, so repeat the GET until status is terminal. For short jobs, POST /v2/actors/{actorId}/run-sync waits up to 45 seconds and returns the OUTPUT key-value record instead of the run object. See Waiting for runs and builds. See Run Actor synchronously.

Fetch dataset items

Use defaultDatasetId from the run:

curl -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  "https://api.scrape.ly/v2/datasets/$DATASET_ID/items"

You can also skip the storage ID and read straight from the run: GET /v2/actor-runs/{runId}/dataset/items takes the same parameters. See Run storages.

Named output (for example OUTPUT) is a record in the run's default key-value store:

curl -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  https://api.scrape.ly/v2/key-value-stores/$KVS_ID/records/OUTPUT

Logs are plain text:

curl -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  https://api.scrape.ly/v2/actor-runs/$RUN_ID/log

Reference

• API overview: auth, pagination, errors, rate limits
• Actors: Actors, versions, builds, starting runs
• Actor runs: inspect, abort, resurrect, logs
• Actor tasks: saved inputs and run configurations
• Datasets · Key-value stores · Request queues
• Automations · Webhooks · Schedules