Tasks, schedules & webhooks

Once an Actor does what you need, automate it. Saved inputs (tasks in the API) keep an input and run options under a name. Automations do everything else: run Actors on a timetable, message you on Slack, Discord, or email when a run finishes, write results to Google Sheets, call your systems with a webhook, or start another Actor after a run. If you build Actors, CI rebuilds them on every git push. Most of this works from the dashboard and the API.

In the dashboard there is one place for all of it: Automations in the side menu, and the Automations tab of an Actor or a saved input. A message or webhook about a run is an automation with two parts: when (a run of an Actor finishes, with the outcomes you pick) and then (one place to send it). This page is the reference for what those messages and webhooks contain; Automations explains the editor.

Tasks

A task is a saved configuration of one Actor: an input plus run options (build, memory, timeout, maximum cost). The dashboard calls tasks saved inputs; see Saved inputs. The build option only applies to your own Actors; someone else's Actor runs its User Build. Tasks are useful for a crawl you repeat with the same settings, for scheduling, and for starting a known configuration from a webhook.

Create a task

• From an Actor: set up the input on the Input tab, then choose Save as new… in the Input dropdown. The saved input keeps the current input and run options.
• From the list: open Actors → Saved inputs in the side menu (or Manage all saved inputs in an Actor's Input dropdown) and click Create.
• From the API: POST /v2/actor-tasks with actId (the Actor's ID), name, and optionally title, input, and options (build, timeoutSecs, memoryMbytes, maxItems, maxTotalChargeUsd, restartOnError). build only applies to your own Actors; someone else's Actor always runs its User Build.

curl -X POST https://api.scrape.ly/v2/actor-tasks \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "actId": "<actorId>",
    "name": "daily-coffee-shops",
    "input": {"searchTerms": ["coffee"], "maxResults": 100},
    "options": {"memoryMbytes": 1024, "timeoutSecs": 900}
  }'

Task names can be up to 30 characters and must be unique in your account (409 otherwise). An empty input ({}) is filled with the defaults from the Actor's input schema. Your plan limits how many tasks you can have (403 when you reach it).

Run a task

Open the task and click Start. From the API:

The request body is optional. Any fields you send are merged over the task's saved input, one level deep, so you can override a single field for one run. Query parameters such as memory, timeout, build (your own Actors only), maxTotalChargeUsd, and waitForFinish work as they do for Actor runs and override the task's options. Only the task's owner can run it.

From the CLI, scrapely task run <taskId> accepts --build (your own Actors only), --memory, --timeout, and --json. The CLI can also publish and unpublish tasks, but you create and edit them in the dashboard or through the API.

See the Tasks API.

Schedules

A schedule starts one or more Actors or saved inputs on a cron timetable. In the dashboard, a schedule is an automation with On a schedule as its trigger: open Automations in the side menu and click New automation. The editor shows the input of each run, lets you change single fields for the automation (Customize), and starts the time zone at your browser's.

The cron expression is evaluated in the schedule's time zone, including daylight saving. For example, 0 9 * * * with Europe/Berlin fires at 09:00 Berlin time all year.

Create a schedule with the API

curl -X POST https://api.scrape.ly/v2/schedules \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "weekday-mornings",
    "cronExpression": "0 7 * * 1-5",
    "isEnabled": true,
    "actions": [
      {
        "type": "RUN_ACTOR_TASK",
        "actorTaskId": "<taskId>"
      },
      {
        "type": "RUN_ACTOR",
        "actorId": "<actorId>",
        "runInput": {"body": "{\"maxResults\": 20}", "contentType": "application/json"},
        "runOptions": {"memoryMbytes": 1024, "timeoutSecs": 600}
      }
    ]
  }'

• name, cronExpression, and at least one action are required. A schedule can hold up to 10 Actor actions and 10 task actions.
• runInput.body is the input as a JSON string; set contentType to application/json so it is parsed. On a RUN_ACTOR action, input is chosen in this order: the action's runInput, then your default saved input, then the schema defaults. On a RUN_ACTOR_TASK action, the runInput fields are merged over the task's input, one level deep.
• runOptions accepts build (your own Actors only; someone else's Actor always runs its User Build), timeoutSecs, memoryMbytes, maxItems, maxTotalChargeUsd, and restartOnError. Missing options come from the task, then from the Actor's defaults.
• isExclusive: true skips a firing while the previous firing's runs are still going.
• notificationsEnabled: false turns off the schedule's failure alert.

Scheduled runs have the origin SCHEDULER and are billed like any other run. A Full Access Actor must already be approved, or its scheduled runs fail to start. GET /v2/schedules/{scheduleId}/log lists recent firings. To test a schedule, use Run now on the automation, or POST /v2/automations/{id}/runs with the schedule's ID. See the Schedules API and the Automations API, which also has filters, an AI step, and deliveries.

Cron syntax

A cron expression has five fields: minute (0–59), hour (0–23), day of month (1–31), month (1–12 or JAN–DEC), and day of week (0–7 or SUN–SAT, where 0 and 7 are Sunday).

If you restrict both day of month and day of week, a day matches if either matches, as in standard cron. For daylight-saving details, see Cron expressions.

Webhooks

A webhook sends an HTTP POST to a URL when an event happens, for example when a run succeeds. Use webhooks to push results into your pipeline instead of polling, or to start another Actor or task when a run finishes.

Set up a webhook in the dashboard

A webhook is an automation: open Automations (or an Actor's Automations tab) and click New automation, then pick Call a webhook when a run finishes. The automation starts out as "when a run of this Actor finishes, send a POST to this URL". You can change:

• When: the Actor, or one of its saved inputs, or every run in your account, and the events (see Event types).
• Then: the URL, an optional payload template and headers template, and the options Ignore SSL errors, Interpolate variables in string fields, and Do not retry on failure.

Such an automation is stored as a webhook, so it is also listed by GET /v2/webhooks. The other way round, every webhook you create through the API, an SDK, or an API client is listed under Automations too. There you can turn it on or off, edit it, send a test, and open its delivery history.

To start another Actor or saved input after a run, use the Start another actor after this one recipe. It creates an automation that runs after another run finishes, which can also wait for several runs, filter, summarize, and send. Its step input can take values from the finished run, such as {{resource.defaultDatasetId}}; see Automations.

Create a webhook with the API

curl -X POST https://api.scrape.ly/v2/webhooks \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "eventTypes": ["ACTOR.RUN.SUCCEEDED", "ACTOR.RUN.FAILED"],
    "condition": {"actorId": "<actorId>"},
    "requestUrl": "https://example.com/hooks/scrapely",
    "description": "Notify pipeline"
  }'

• eventTypes and requestUrl are required.
• condition limits the webhook to one Actor (actorId), one task (actorTaskId), or one run (actorRunId). Without a condition, the webhook fires for every matching event in your account.
• Optional fields: payloadTemplate, headersTemplate, description, ignoreSslErrors, doNotRetry, shouldInterpolateStrings, isEnabled (false creates it turned off), and idempotencyKey (repeating a create with the same key returns the existing webhook).
• The URL must be reachable from the internet. Private, local, and internal-network addresses are rejected. The Scrape.ly API itself is allowed, so a webhook can start another run.
• Your plan limits the number of webhooks (100 by default).

See the Webhooks API.

Event types

Run events fire for your own runs, including runs of other people's Actors. Build events fire only for Actors you own, and setting them on someone else's Actor is rejected with 400 invalid-input.

Payload

By default the request body is:

{
  "userId": "<your user ID>",
  "createdAt": "2026-10-12T09:00:00.000Z",
  "eventType": "ACTOR.RUN.SUCCEEDED",
  "eventData": {
    "actorId": "...",
    "actorTaskId": null,
    "actorRunId": "...",
    "status": "SUCCEEDED",
    "startedAt": "...",
    "finishedAt": "...",
    "durationMillis": 41234,
    "defaultDatasetId": "...",
    "defaultKeyValueStoreId": "...",
    "defaultRequestQueueId": "...",
    "buildId": "...",
    "exitCode": 0,
    "computeUnits": 0.012
  },
  "resource": { "...": "same fields as eventData" }
}

On runs of other people's Actors, buildId is left out: build info is owner only.

For build events, eventData has actorId, buildId, status, versionNumber, buildTag, exitCode, and timing fields.

Payload and header templates

A payloadTemplate replaces the default body. These variables are available:

• {{userId}}, {{createdAt}}, {{eventType}}, {{eventData}}, {{resource}}, and {{globals}} (with dateISO and dateUnix).
• Dotted paths of any depth: {{eventData.actorRunId}}, {{resource.defaultDatasetId}}, {{resource.stats.computeUnits}}, and so on.
• Shortcuts: {{actorId}}, {{actorRunId}}, {{status}}, {{startedAt}}, {{finishedAt}}, {{buildId}}, {{defaultDatasetId}}, {{defaultKeyValueStoreId}}, {{exitCode}}, {{durationMillis}}, {{computeUnits}}.

Outside quotation marks, each variable is replaced by its JSON value, and strings include their quotes:

{
  "text": "Run finished",
  "runId": {{actorRunId}},
  "status": {{status}},
  "datasetId": {{resource.defaultDatasetId}}
}

Inside a JSON string, a variable that is the whole string ("{{resource.defaultDatasetId}}") is replaced by its JSON value, and a variable inside other text ("Run {{actorRunId}} done") by its plain text. shouldInterpolateStrings: false leaves strings as written.

A headersTemplate is a JSON object of extra request headers. Inside header values you can use {{eventType}}, {{actorId}}, {{actorRunId}}, {{status}}, {{userId}}, and {{createdAt}}, inserted as plain text. Setting shouldInterpolateStrings to false turns off variables in headers.

Delivery, retries, and verification

Every request carries Content-Type: application/json, User-Agent: Scrapely-Webhook/1.0, and an X-Scrapely-Webhook-Dispatch-Id header that stays the same across retries of one delivery, so you can use it to ignore duplicates.

• Your endpoint has 30 seconds to respond. Redirects are not followed.
• Network errors, timeouts, 5xx, and 429 responses are retried up to 5 times, after 1, 2, 4, 8, and 16 seconds. Other 4xx responses are not retried. doNotRetry: true turns retries off.
• To check that a request really comes from your webhook, put a secret header in the headersTemplate (for example {"X-Webhook-Token": "<long random string>"}) and verify it on your server.

Each delivery is recorded as a dispatch with its status code and response. Up to 500 recent dispatches are kept per webhook.

Ad-hoc webhooks

An ad-hoc webhook is created for a single use and deleted after it fires once.

When you start a run through the API, pass ad-hoc webhooks in the webhooks query parameter as a base64-encoded JSON array:

WEBHOOKS=$(echo -n '[{"eventTypes":["ACTOR.RUN.SUCCEEDED"],"requestUrl":"https://example.com/hook"}]' | base64)

curl -X POST "https://api.scrape.ly/v2/actors/<actorId>/runs?webhooks=$WEBHOOKS" \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}'

You can also send a webhooks array in the JSON body. Each ad-hoc webhook is bound to the run it was sent with, so it never fires for another run of the same Actor or task. While its run is active, it counts toward your plan's webhook limit; past the limit the run start returns 403 limit-reached and no run is created. A run token of a limited-permission Actor can't attach ad-hoc webhooks (403 insufficient-permissions).

From inside an Actor, Actor.addWebhook() (JavaScript) or Actor.add_webhook() (Python) registers an ad-hoc webhook for the current run. See JavaScript SDK and Python SDK.

CI/CD: rebuild on git push

For Actors whose source is a Git repository, Scrape.ly can start a new latest build on every push.

If you've connected GitHub (see below), this is set up for you. When you link a version to a GitHub repository, Scrape.ly adds a push webhook to that repository. This happens whether you create the Actor, change its versions, or point a version at a new repository, in the dashboard or through the API with your API key. If an older webhook for the same Actor and version is already there, for example one that carries your API key instead of the build secret, Scrape.ly updates it instead of adding a second one. Linking never fails because of the webhook; if GitHub refuses it, add it by hand as described below.

A push only starts a build when it comes from the linked repository and updates the linked branch: the #branch in the repository URL, or the repository's default branch if the URL has none. A tag with that name also counts. Other pushes are answered with 200 and no build starts, so pushes to feature branches don't spend build minutes. See Start a build for the response.

To set it up by hand, open the Actor, go to the Settings tab, and find Continuous integration (GitHub), or fetch the build URL from the API:

curl -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  "https://api.scrape.ly/v2/actors/<actorId>/ci/webhook-url"

The returned webhookUrl contains a build secret for that Actor only, not your API token. Pass version=<versionNumber> to target a specific version; by default it uses the Actor's Git-repository version. Then either:

• GitHub webhook: in the repository, open Settings → Webhooks → Add webhook, paste the URL as the Payload URL, set the content type to application/json, and select the push event.
• GitHub Actions: store the URL as a repository secret named BUILD_ACTOR_URL and add a workflow step:

- name: Trigger Scrapely build
  run: curl -fsS -X POST "$BUILD_ACTOR_URL"
  env:
    BUILD_ACTOR_URL: ${{ secrets.BUILD_ACTOR_URL }}

Each request pulls the latest code from the linked repository and builds it. The GitHub Actions call has no X-GitHub-Event header, so the branch filter doesn't apply; choose the branches in the workflow's on: trigger.

To add or repair the webhook for an Actor you linked before automatic registration, call POST /v2/actors/{actorId}/ci/webhook with an empty body. It registers the webhook on the linked repository, or fixes the existing one. See CI webhooks. scrapely create --source github --auto-build on also registers it for new projects. See CLI CI/CD for running CLI commands in a pipeline.

GitHub and Slack accounts

Connect accounts under Settings → Connections:

• GitHub lets you create Actors from your repositories, and registers a push webhook on each repository you link so pushes rebuild the Actor. GET /v2/integrations/github/repos lists the repositories Scrape.ly can see.
• Slack connects a workspace and one channel. Slack messages from your automations go there. You can also point an HTTP webhook at a Slack incoming-webhook URL with your own payload template.

GET /v2/integrations returns the connection status of slack and github. DELETE disconnects one. See Search, alerts & utilities.

Slack, Discord and Email

An automation that reacts to a run and sends to Slack, Discord, or email posts a ready-made message about that run. Create one with New automation and the recipe Notify me on Slack when a run fails or Email me the results, then change the channel or the outcomes if you like. For one message about several runs, with filters or an AI summary, add run steps to the automation instead. Each one has:

• Send a message when a run: succeeded, failed, timed out, or was aborted.
• Only for runs with results: skips successful runs that returned no items. Failed runs are always sent.
• Results preview: the first 0 to 5 dataset items, shown as JSON.
• Custom message (optional): shown above the run details. Variables: {{actorName}}, {{taskName}}, {{status}}, {{runId}}, {{itemCount}}, {{duration}}, {{cost}}, {{runUrl}}, {{datasetUrl}}.

Every message shows the Actor, status, duration, result count and cost, with links to the run and its dataset in the dashboard. Send a test from the automation's page to check the setup with sample data. The automation's page also shows the delivery history, with the reason for any failed delivery.

Where messages go

• Slack: the channel you picked when you connected Slack under Settings → Connections. Change channel reconnects Slack; the new channel applies to all your Slack automations.
• Discord: a channel webhook URL. In Discord, open the channel's settings → Integrations → Webhooks → New Webhook → Copy Webhook URL. Only https://discord.com/api/webhooks/… URLs are accepted. The URL is stored encrypted and shown masked afterwards. Messages never ping @everyone or users.
• Email: your account email works right away. Any other address gets a confirmation link first; nothing is sent to it until the link is opened. Every email has an unsubscribe link, and turning off email notifications under Settings → Notifications stops them too.

Limits: each Slack or Discord automation of this kind sends at most 20 messages a minute; each email one at most 30 emails an hour, and an account at most 100 of these emails a day. Messages over a limit are logged as failed deliveries. Slack and Discord deliveries are retried on rate limits, server errors and timeouts.

API: these automations are webhooks with a notification object instead of a requestUrl, and are listed under /v2/webhooks and /v2/automations?include=webhooks:

curl -X POST https://api.scrape.ly/v2/webhooks \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "eventTypes": ["ACTOR.RUN.SUCCEEDED", "ACTOR.RUN.FAILED"],
    "condition": { "actorId": "YOUR_ACTOR_ID" },
    "description": "Discord alerts",
    "notification": {
      "channel": "discord",
      "discordWebhookUrl": "https://discord.com/api/webhooks/…",
      "onlyWithResults": true,
      "previewItems": 3,
      "message": "{{actorName}}: {{itemCount}} results"
    }
  }'

channel is slack, discord, or email (with email). Responses carry notification with a masked target; the Discord URL is never returned. POST /v2/webhooks/{id}/test sends a test message. Only the run-finished events are allowed (ACTOR.RUN.SUCCEEDED, FAILED, ABORTED, TIMED_OUT). POST /v2/integrations/notifications/email/verify with { "email": "…" } sends a confirmation link, and GET /v2/integrations/notifications/email lists your addresses and their state.

Google Sheets

Writes the dataset items of each successful run into a Google Sheet. There is no Google sign-in: Scrape.ly writes as its own service account, and you give it access to just the sheets you choose.

1. Click New automation and pick Send results to Google Sheets (on an Actor's Automations tab the Actor is already chosen).
2. Copy the email shown under Share your sheet with. In Google Sheets, click Share, add that email and give it Editor access.
3. Paste the sheet's link, and optionally the Tab to write to (the first tab if empty).
4. Pick what happens On each run:
   • Add rows: new results go below the existing rows.
   • Replace the tab: the tab is cleared and filled with the latest results.
5. Save and send a test. A Scrapely connected ✓ row appears in the sheet; the first real run puts its header in that row.

Columns: the first row is a header. Nested fields become columns named with dots, such as price.amount; arrays are written as JSON text. With Add rows, columns already in the header keep their position and new fields are added at the end. Values are written as plain text and numbers, so text that starts with = is never run as a formula. A cell holds at most 50,000 characters.

When it writes: only for succeeded runs, and only when the run returned items. Max rows per run (1 to 10,000, default 1,000) caps how many items are written.

Limits: each Google Sheets automation of this kind writes at most 30 times an hour, and an account at most 500 times a day. If the sheet isn't shared with the service account, or the tab doesn't exist, the delivery history on the automation's page says so.

API: a webhook with a gsheets notification. Only ACTOR.RUN.SUCCEEDED is allowed.

curl -X POST https://api.scrape.ly/v2/webhooks \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "eventTypes": ["ACTOR.RUN.SUCCEEDED"],
    "condition": { "actorId": "YOUR_ACTOR_ID" },
    "description": "Results to Google Sheets",
    "notification": {
      "channel": "gsheets",
      "sheetUrl": "https://docs.google.com/spreadsheets/d/…/edit",
      "sheetName": "Results",
      "mode": "append",
      "maxRows": 1000
    }
  }'

mode is append or replace, and sheetName may be null for the first tab. GET /v2/integrations/gsheets returns serviceAccountEmail, the address to share sheets with.

Notifications

Email and dashboard alerts for failed runs, storage expiry, low balance, and more are configured under Settings → Notifications. See Notifications.

What's next

• Run Actors: run options and limits
• Results & exports: what to fetch when a webhook fires
• Automations: schedules, messages, webhooks, filters, an AI step, and deliveries in one place
• Automations API, Webhooks API, Schedules API, and Tasks API