How Actors work

An Actor is a program that runs in a container on Scrape.ly. You write the code and an input schema; the platform builds the image, runs it on demand or on a schedule, stores the results, and bills the compute.

import { Actor } from 'scrapely';

await Actor.main(async () => {
    const input = await Actor.getInput();
    await Actor.pushData({ received: input });
});

Actor.main() initializes the SDK, runs your function, and exits with the right status. Don't also wrap the callback in async with Actor:.

What is an Actor?

One unit of automation: usually a scraper, but any job works, such as enriching records, monitoring prices, or processing files.

Project layout

The platform uses the first Actor config it finds:

1. .actor/actor.json
2. .actor/scrapely.json
3. scrapely.json
4. actor.json

If a project has both .actor/ config and a root scrapely.json, fields in the root file override the .actor/ ones.

{
    "actorSpecification": 1,
    "name": "my-actor",
    "title": "My Actor",
    "version": "0.0",
    "buildTag": "latest",
    "input": "./input_schema.json",
    "storages": { "dataset": "./dataset_schema.json" },
    "environmentVariables": { "API_KEY": "@myApiKey" },
    "minMemoryMbytes": 512,
    "maxMemoryMbytes": 4096
}

Config fields

Without input, the platform looks for .actor/input_schema.json, .actor/INPUT_SCHEMA.json, input_schema.json, INPUT_SCHEMA.json, then schema.json. Paths resolve relative to the config file.

Check every schema the config references before you push:

scrapely validate-schema

Lifecycle

1. The platform starts your container and injects the run environment: run token, run ID, default storage IDs, timeout, memory, proxy settings, and CDP_URL for the Scrape.ly browser. See JavaScript environment and Python environment.
2. Actor.init(), which Actor.main() calls for you, connects storage, events, and charging.
3. Your code reads input, does the work, and writes to storage.
4. Actor.exit() flushes pending writes and exits. If the process crashes, the run is FAILED and its log is kept.

The SDK doesn't start an HTTP server. If you need one, for Standby for example, listen on the port the platform gives you.

Input

Input is the JSON object your Actor gets at the start of a run. The input schema describes it and drives the dashboard form:

{
    "title": "Listing scraper input",
    "type": "object",
    "schemaVersion": 1,
    "properties": {
        "startUrl": {
            "title": "Start URL",
            "type": "string",
            "description": "Where the crawl begins",
            "editor": "textfield",
            "prefill": "https://example.com"
        },
        "maxItems": { "title": "Max items", "type": "integer", "default": 50, "minimum": 1 },
        "proxyConfiguration": {
            "title": "Proxy",
            "type": "object",
            "editor": "proxy",
            "default": { "useScrapelyProxy": true }
        }
    },
    "required": ["startUrl"]
}

Other editor values are accepted and fall back to the control for the field's type. Users can always switch to a raw JSON editor.

• "isSecret": true masks the value in saved input and encrypts it in the run's stored input. See Security.
• "editor": "resourcePicker" with a resourceType lets a limited-permission Actor access the storage IDs a user passes in.
• An input schema can be up to 500 KB.
• The form shows fields in the order properties lists them, so put the important ones first.

Group fields into categories

An Actor with many options can keep the form short: leave the 1–3 main inputs ungrouped and put the rest into collapsible categories. Categories only change how the form looks. They don't change the input, defaults, validation or runs.

{
    "title": "Google Search Scraper",
    "type": "object",
    "schemaVersion": 1,
    "categories": {
        "filters": {
            "title": "Search filters",
            "description": "Narrow results by site, words and dates.",
            "defaultCollapsed": true
        },
        "dates": { "title": "Date range", "parent": "filters" },
        "ads-related": { "title": "Related to Ads", "defaultCollapsed": true }
    },
    "properties": {
        "queries": { "title": "Search queries", "type": "string", "editor": "textarea" },
        "site": { "title": "Site", "type": "string", "editor": "textfield", "category": "filters" },
        "afterDate": { "title": "After date", "type": "string", "editor": "textfield", "category": "dates" },
        "liveAiOverview": { "title": "Live AI overview", "type": "boolean", "category": "ads-related" }
    },
    "required": ["queries"]
}

Here queries is the main input, Search filters starts collapsed, and Date range sits inside it.

• Fields without a category come first, then each category with its fields in schema order, then its nested categories.
• Categories nest up to 3 levels deep. A deeper category is shown at level 3. A parent that loops back (a → b → a) or names no category puts the category at the top level.
• A category with no fields, directly or through nested categories, isn't shown.
• A schema can define up to 100 categories. With more, categories is ignored and every field is shown ungrouped.
• Each section header shows how many fields it holds and how many are set to a non-default value. A section opens on its own when one of its fields has an error or is required and empty.
• sectionCaption and sectionDescription also work: a field with a caption starts a collapsed section that runs to the next caption. These sections come after your categories. A field with a category ignores sectionCaption.
• A category that names no defined category doesn't fail the build. The build log warns, with the closest key (unknown category "ad-related". Did you mean "ads-related"?), and the field is shown ungrouped. scrapely validate-schema treats it as an error, along with parent loops, nesting deeper than 3 and wrong types, and warns about unused categories.

Read input with Actor.getInput() / Actor.get_input(), or getInputOrThrow() / get_input_or_throw() when missing input should fail the run.

Output

Results go to storage, not to files in the container:

Each run has its own default dataset, key-value store, and request queue. See Storage.

Storage schemas

storages in the config attaches schemas to a run's storages and can create extra ones:

{
    "storages": {
        "dataset": "./dataset_schema.json",
        "keyValueStores": {
            "default": "./kvs_schema.json",
            "screenshots": "./screenshots_schema.json"
        }
    }
}

• Open an extra storage by alias: Actor.openKeyValueStore({ alias: 'screenshots' }) (JavaScript) or Actor.open_key_value_store(alias="screenshots") (Python). The aliases reach the run in ACTOR_STORAGES_JSON.
• A dataset schema with fields (JSON Schema draft-07) validates every pushed item. If one fails, nothing is stored and the push returns 400 schema-validation-error. See Datasets API.

Output schema

An output schema tells users where a run's results are. Each property's template becomes a link on the run's Output tab and in the run object's output map:

{
    "actorOutputSchemaVersion": 1,
    "title": "Output",
    "properties": {
        "results": {
            "type": "string",
            "title": "Results",
            "template": "{{links.apiDefaultDatasetUrl}}/items"
        },
        "screenshots": {
            "type": "string",
            "title": "Screenshots",
            "template": "{{storages.keyValueStores.screenshots.apiUrl}}/keys"
        }
    }
}

Unknown variables resolve to an empty string. The dashboard shows dataset links as a table and key-value store keys or records/<key> links as their contents.

Versions

An Actor has one or more versions, numbered MAJOR.MINOR (0.0, 0.1, 1.0). Each version has:

• A source: uploaded files, a GitHub repository, or a tarball URL (up to 500 MB). Git sources accept https://github.com/... and git@github.com:... URLs only. A version's stored source can hold up to 10,000 files and 128 MB; see Versions for the upload limits.
• Environment variables, plain or secret
• A build tag (default latest) for its successful builds

scrapely push uploads the current directory as the source of the version in your config.

GET returns the version with its file contents, so you can read a version, change a field, and PUT it back without touching the source. See the Actors API.

Builds

A build turns one version's source into a Docker image. Only the owner can build. Builds start from:

• scrapely push
• The dashboard, on the Actor's Builds tab
• The API: POST /v2/actors/:actorId/builds?version=0.0
• The Actor's CI webhook, for Git versions. It pulls the latest commit first, and the build reads actor.json and the input schema from that commit.

Base images

Without a Dockerfile, the platform generates one: Node.js projects install package.json dependencies and start main.js, Python projects start main.py. With your own Dockerfile, start FROM a Scrape.ly base image, which already contains the SDK:

Tag the image with the version, for example scrapely/actor-node-playwright-chrome:22. Builds also accept the matching apify/actor-* images, but publishing to the Store requires a scrapely/actor-* base.

Options and limits

Pass options in the request body as {"options": {...}}:

Build memory counts toward your account's total memory limit. Your plan's limits are under Billing → Limits.

Statuses and tags

• A build goes READY → RUNNING → SUCCEEDED, FAILED, TIMED-OUT, or ABORTED.
• A successful build gets its version's build tag (latest by default). latest always belongs to the newest succeeded build by start time (ties go to the higher build number), not to the build that finished last: a build that finishes after a newer one already holds latest stays untagged. If the holder leaves SUCCEEDED, latest moves back to the newest succeeded build. You can add your own tags: 1 to 64 letters, digits, ., _, and -, starting with a letter or digit.
• Runs use latest unless you pass build or the Actor has a default build. Other users can't pick a build: they always run the User Build.
• Each build records whether it uses the Scrape.ly browser (browserPatched).

Builds, build logs, and versions are owner only, also for public Actors.

Actor settings

Per-Actor settings live on the Actor's Settings tab in the dashboard and on the Actor object in the API. Only the owner can change them.

Store listing fields (title, description, SEO, categories, icon, pricing, source visibility) and the User Build (the build other users run, defaultNonOwnerBuild) are on the Publication tab. See Publish and monetize.

Runs

A run executes one build with one input, and needs at least one successful build. Start runs from:

• The dashboard
• The CLI: scrapely call for a platform run, scrapely run for the local project on your machine
• The API: POST /v2/actors/:actorId/runs
• Tasks and schedules

Statuses: READY, RUNNING, SUCCEEDED, FAILED, ABORTING, ABORTED, TIMING-OUT, TIMED-OUT.

Run options

Query parameters on POST /v2/actors/:actorId/runs. Anything you leave out falls back to the Actor's defaultRunOptions, then the platform default:

Send an Idempotency-Key header to make a retried start return the same run instead of starting another. See Idempotent run starts.

The request body is the run's input.

run-sync and run-sync-get-dataset-items start a run and wait for it, for at most 45 seconds. After that they return 408 with the run ID, and the run keeps going. Without timeout, they cap the run at 300 seconds. See Waiting for runs and builds.

Read a run's log with GET /v2/actor-runs/:runId/log or scrapely runs log <runId>. See the Runs API.

Standby

A Standby Actor keeps a container warm and serves HTTP requests instead of batch runs:

ALL /v2/:username/:actorName/*

Callers send their own Bearer token and are billed for the run. The Actor must be public, or the caller must own it. Configure it under actorStandby or on the Standby Mode settings card:

Listen on ACTOR_WEB_SERVER_PORT and bind to 0.0.0.0. Read the port with Actor.getEnv().containerPort (JavaScript) or Actor.get_env().container_port (Python). Standby runs have ACTOR_META_ORIGIN=STANDBY and ACTOR_STANDBY_URL. See Standby API and the JavaScript Standby guide.

Publishing

To list an Actor in the Store it needs a title, a description, a README, a category, a successful build, and an input schema. A Dockerfile must use a scrapely/actor-node (Node.js 20, 22, 24, or 26) or scrapely/actor-python (Python 3.11 or later) base. Most Actors go live immediately; some get a short manual review first. Set pay-per-event pricing and earn 80% of what users pay with purchased credits, paid out when you request it.

See Publish and monetize.

Full example

import { Actor, CheerioCrawler } from 'scrapely';

await Actor.main(async () => {
    const { startUrl = 'https://news.ycombinator.com', maxItems = 25 } =
        (await Actor.getInput()) ?? {};

    const proxyConfiguration = await Actor.createProxyConfiguration();

    const crawler = new CheerioCrawler({
        proxyConfiguration,
        maxRequestsPerCrawl: maxItems,
        requestHandler: async ({ $, request }) => {
            await Actor.pushData({
                url: request.url,
                title: $('title').text().trim(),
            });
        },
    });

    await crawler.run([startUrl]);
});

Push it with scrapely push and run it with scrapely call. The results land in the run's default dataset, ready to export from the dashboard, the CLI, or the API.