Build Actors

Learning path

The order to read the Build docs in, with walkthroughs from a first Actor to a paid listing.

This learning path picks up after the quick start and takes you from a first Actor to a published, paid one. Each step is a page in these docs, not a hosted course. You write code on your machine, try it locally with scrapely run, then deploy with scrapely push and run it on the platform with scrapely call.

The path

Ship your first Actor. Install the CLI, create a project, push it, and run it. Quick start
Understand Actors. Learn the project layout, input schema, versions, builds, runs, and Standby. Actors
Store results. Use datasets, key-value stores, and request queues, and learn how long they're kept. Storage
Avoid blocks. Use the managed residential and datacenter proxies, your own proxies, and the opt-in Scrape.ly browser. Proxy, Scrape.ly Browser
Keep it safe. Protect tokens and secrets, and choose a permission level. Security
Go deeper in your language. JavaScript SDK or Python SDK
Work from the terminal. CLI
Publish and earn. Create a Store listing, set pay-per-event pricing, and track earnings. Publish and monetize

When you want to automate runs or call Actors from your own code, continue with Integrations and the API.

Guided walkthroughs

Work through these in order. Each one ends with a run on the platform.

Scrape a listing into a dataset

Start from the Cheerio template (JavaScript) or the BeautifulSoup template (Python):

.bash

scrapely create listing-scraper --template crawlee-cheerio-js
cd listing-scraper

For TypeScript, use --template ts-crawlee-cheerio. Give the input schema a startUrl and a maxItems field with default values, so the dashboard form is ready to run. See Actors.

In the request handler, push one object per row with Actor.pushData() / Actor.push_data(). Try it locally, then deploy and run it:

.bash

scrapely run -i '{"startUrl":"https://example.com","maxItems":10}'
scrapely push
scrapely call -i '{"startUrl":"https://example.com","maxItems":10}' -o

scrapely run writes to ./storage on your machine. scrapely call runs the Actor that scrapely push saved in your project config, and -o prints the run's dataset when it finishes. To export a CSV, take the run's defaultDatasetId and run:

.bash

scrapely datasets get-items <datasetId> --format csv > results.csv

See Storage.

Scrape a JavaScript-heavy site

Use a Playwright template: crawlee-playwright or crawlee-playwright-ts. Import PlaywrightCrawler from scrapely.

By default the crawler launches Chromium inside the container, and you route it with Actor.createProxyConfiguration() like any other crawler. To offload the browser instead, turn on Use Scrapely browser in the Actor's settings and rebuild. Platform runs then connect to the hosted Scrape.ly browser, which has its own residential proxy and Cloudflare solving, and bills browser credits. Set its country and fingerprint with Actor.createBrowserConfiguration(). Locally, the crawler always launches a normal browser. See Proxy.

Use the managed proxy

For HTTP crawlers, pass await Actor.createProxyConfiguration() to the crawler. On the platform, that sends traffic through the residential pool. Use groups: ['DATACENTER'] for the cheaper datacenter pool. To let users choose, add a field with "editor": "proxy" to the input schema and pass its value to the SDK. See Proxy.

Wrap an existing script

In an existing Node.js project, run scrapely init. It adds scrapely.json, plus package.json and main.js if they're missing. In a Scrapy project (one with scrapy.cfg), it also writes a main.py wrapper. Without a Dockerfile, the platform generates one when it builds.

Wrap your entrypoint in Actor.main() (JavaScript) or asyncio.run(Actor.main(main)) (Python). Don't put async with Actor: inside Actor.main().

Replace file writes with pushData / setValue so results land in the run's dataset and key-value store. Then run scrapely push and scrapely call.

Schedule a recurring run

Save an Actor with a fixed input as a task, then attach a cron schedule. The CLI can run tasks (scrapely task run <taskId>) but can't create tasks or schedules. Create them in the dashboard under Actors → Saved inputs and Automations, or through the API with scrapely api:

.bash

scrapely api POST /v2/actor-tasks \
  -d '{"actId":"<actorId>","name":"nightly","input":{"maxItems":100}}'

scrapely api POST /v2/schedules \
  -d '{"name":"nightly","cronExpression":"0 2 * * *","timezone":"UTC","isEnabled":true,"actions":[{"type":"RUN_ACTOR_TASK","actorTaskId":"<taskId>"}]}'

Task names are up to 30 characters. Each trigger creates a normal run, with its own logs and storage. See Integrations.

Deliver results with a webhook

Have Scrape.ly call your endpoint when a run succeeds. Use a publicly reachable URL (HTTPS recommended), because localhost and private-network URLs are rejected. Create it on the Actor's Automations tab with New automation → Call a webhook when a run finishes, or through the API:

.bash

scrapely api POST /v2/webhooks \
  -d '{"eventTypes":["ACTOR.RUN.SUCCEEDED"],"condition":{"actorId":"<actorId>"},"requestUrl":"https://example.com/hooks/scrapely?key=<your-secret>"}'

In your handler, check your secret, read resource.defaultDatasetId from the payload, and fetch the items with GET /v2/datasets/:datasetId/items. See the Webhooks API and Security.

Expose an Actor as an HTTP API

Turn on Standby for the Actor (actorStandby.isEnabled). Your process has to listen on the port from Actor.getEnv().containerPort (JavaScript) or Actor.get_env().container_port (Python), bound to 0.0.0.0. The SDK doesn't start a server for you, and you shouldn't call Actor.exit() while handling a request.

Callers send requests to https://api.scrape.ly/v2/<username>/<actorName>/... with their own Bearer token. See the JavaScript Standby guide, Python environment, and the Standby API.

Publish and charge for it

Add a README, an input schema with working defaults, and a Store description. Then publish from the Publication tab. Set pay-per-event pricing, call Actor.charge() for your custom events, and stop when eventChargeLimitReached is true. See Publish and monetize.

Example projects

Browse public Actors in the Store. Run one, read its README and input schema, then build your own version from a template with scrapely create.

Keep going

JavaScript SDK and Python SDK
CLI reference
When something fails, start with CLI configuration and the deploy guides for JavaScript and Python