--- title: "Learning path · Scrape.ly Docs" description: "The order to read the Build docs in, with walkthroughs from a first Actor to a paid listing." url: https://scrape.ly/docs/academy markdown: https://scrape.ly/docs/academy.md --- # Learning path > The order to read the Build docs in, with walkthroughs from a first Actor to a paid listing. This learning path picks up after the quick start and takes you from a first Actor to a published, paid one. Each step is a page in these docs, not a hosted course. You write code on your machine, try it locally with `scrapely run`, then deploy with `scrapely push` and run it on the platform with `scrapely call`. ## The path 1. **Ship your first Actor.** Install the CLI, create a project, push it, and run it. [Quick start](https://scrape.ly/docs/quickstart.md) 2. **Understand Actors.** Learn the project layout, input schema, versions, builds, runs, and Standby. [Actors](https://scrape.ly/docs/actors.md) 3. **Store results.** Use datasets, key-value stores, and request queues, and learn how long they're kept. [Storage](https://scrape.ly/docs/storage.md) 4. **Avoid blocks.** Use the managed residential and datacenter proxies, your own proxies, and the opt-in Scrape.ly browser. [Proxy](https://scrape.ly/docs/proxy.md), [Scrape.ly Browser](https://scrape.ly/docs/browser.md) 5. **Keep it safe.** Protect tokens and secrets, and choose a permission level. [Security](https://scrape.ly/docs/security.md) 6. **Go deeper in your language.** [JavaScript SDK](https://scrape.ly/docs/sdk-node.md) or [Python SDK](https://scrape.ly/docs/sdk-python.md) 7. **Work from the terminal.** [CLI](https://scrape.ly/docs/cli.md) 8. **Publish and earn.** Create a Store listing, set pay-per-event pricing, and track earnings. [Publish and monetize](https://scrape.ly/docs/publishing.md) When you want to automate runs or call Actors from your own code, continue with [Integrations](https://scrape.ly/docs/integrations.md) and the [API](https://scrape.ly/docs/api-overview.md). ## Guided walkthroughs Work through these in order. Each one ends with a run on the platform. ### Scrape a listing into a dataset Start from the Cheerio template (JavaScript) or the BeautifulSoup template (Python): ```bash scrapely create listing-scraper --template crawlee-cheerio-js cd listing-scraper ``` For TypeScript, use `--template ts-crawlee-cheerio`. Give the input schema a `startUrl` and a `maxItems` field with `default` values, so the dashboard form is ready to run. See [Actors](https://scrape.ly/docs/actors.md). In the request handler, push one object per row with `Actor.pushData()` / `Actor.push_data()`. Try it locally, then deploy and run it: ```bash scrapely run -i '{"startUrl":"https://example.com","maxItems":10}' scrapely push scrapely call -i '{"startUrl":"https://example.com","maxItems":10}' -o ``` `scrapely run` writes to `./storage` on your machine. `scrapely call` runs the Actor that `scrapely push` saved in your project config, and `-o` prints the run's dataset when it finishes. To export a CSV, take the run's `defaultDatasetId` and run: ```bash scrapely datasets get-items --format csv > results.csv ``` See [Storage](https://scrape.ly/docs/storage.md). ### Scrape a JavaScript-heavy site Use a Playwright template: `crawlee-playwright` or `crawlee-playwright-ts`. Import `PlaywrightCrawler` from `scrapely`. By default the crawler launches Chromium inside the container, and you route it with `Actor.createProxyConfiguration()` like any other crawler. To offload the browser instead, turn on **Use Scrapely browser** in the Actor's settings and rebuild. Platform runs then connect to the hosted [Scrape.ly browser](https://scrape.ly/docs/browser.md), which has its own residential proxy and Cloudflare solving, and bills browser credits. Set its country and fingerprint with `Actor.createBrowserConfiguration()`. Locally, the crawler always launches a normal browser. See [Proxy](https://scrape.ly/docs/proxy.md#browser-crawlers). ### Use the managed proxy For HTTP crawlers, pass `await Actor.createProxyConfiguration()` to the crawler. On the platform, that sends traffic through the residential pool. Use `groups: ['DATACENTER']` for the cheaper datacenter pool. To let users choose, add a field with `"editor": "proxy"` to the input schema and pass its value to the SDK. See [Proxy](https://scrape.ly/docs/proxy.md). ### Wrap an existing script In an existing Node.js project, run `scrapely init`. It adds `scrapely.json`, plus `package.json` and `main.js` if they're missing. In a Scrapy project (one with `scrapy.cfg`), it also writes a `main.py` wrapper. Without a Dockerfile, the platform generates one when it builds. Wrap your entrypoint in `Actor.main()` (JavaScript) or `asyncio.run(Actor.main(main))` (Python). Don't put `async with Actor:` inside `Actor.main()`. Replace file writes with `pushData` / `setValue` so results land in the run's dataset and key-value store. Then run `scrapely push` and `scrapely call`. > [!WARNING] > The Python SDK comes preinstalled in the Scrape.ly base images. Don't add `scrapely` to `requirements.txt`. On PyPI, that name belongs to an unrelated library. ### Schedule a recurring run Save an Actor with a fixed input as a task, then attach a cron schedule. The CLI can run tasks (`scrapely task run `) but can't create tasks or schedules. Create them in the dashboard under **Actors** → **Saved inputs** and **Automations**, or through the API with `scrapely api`: ```bash scrapely api POST /v2/actor-tasks \ -d '{"actId":"","name":"nightly","input":{"maxItems":100}}' scrapely api POST /v2/schedules \ -d '{"name":"nightly","cronExpression":"0 2 * * *","timezone":"UTC","isEnabled":true,"actions":[{"type":"RUN_ACTOR_TASK","actorTaskId":""}]}' ``` Task names are up to 30 characters. Each trigger creates a normal run, with its own logs and storage. See [Integrations](https://scrape.ly/docs/integrations.md). ### Deliver results with a webhook Have Scrape.ly call your endpoint when a run succeeds. Use a publicly reachable URL (HTTPS recommended), because localhost and private-network URLs are rejected. Create it on the Actor's **Automations** tab with **New automation** → **Call a webhook when a run finishes**, or through the API: ```bash scrapely api POST /v2/webhooks \ -d '{"eventTypes":["ACTOR.RUN.SUCCEEDED"],"condition":{"actorId":""},"requestUrl":"https://example.com/hooks/scrapely?key="}' ``` In your handler, check your secret, read `resource.defaultDatasetId` from the payload, and fetch the items with `GET /v2/datasets/:datasetId/items`. See the [Webhooks API](https://scrape.ly/docs/api-webhooks.md) and [Security](https://scrape.ly/docs/security.md#webhook-security). ### Expose an Actor as an HTTP API Turn on Standby for the Actor (`actorStandby.isEnabled`). Your process has to listen on the port from `Actor.getEnv().containerPort` (JavaScript) or `Actor.get_env().container_port` (Python), bound to `0.0.0.0`. The SDK doesn't start a server for you, and you shouldn't call `Actor.exit()` while handling a request. Callers send requests to `https://api.scrape.ly/v2///...` with their own Bearer token. See the [JavaScript Standby guide](https://scrape.ly/docs/sdk-node-standby.md), [Python environment](https://scrape.ly/docs/sdk-python-environment.md), and the [Standby API](https://scrape.ly/docs/api-standby.md). ### Publish and charge for it Add a README, an input schema with working defaults, and a Store description. Then publish from the **Publication** tab. Set pay-per-event pricing, call `Actor.charge()` for your custom events, and stop when `eventChargeLimitReached` is true. See [Publish and monetize](https://scrape.ly/docs/publishing.md). ## Example projects Browse public Actors in the [Store](https://scrape.ly/docs/store.md). Run one, read its README and input schema, then build your own version from a template with `scrapely create`. ## Keep going - [JavaScript SDK](https://scrape.ly/docs/sdk-node.md) and [Python SDK](https://scrape.ly/docs/sdk-python.md) - [CLI reference](https://scrape.ly/docs/cli.md) - When something fails, start with [CLI configuration](https://scrape.ly/docs/cli-config.md) and the deploy guides for [JavaScript](https://scrape.ly/docs/sdk-node-deploy.md) and [Python](https://scrape.ly/docs/sdk-python-deploy.md) --- - Previous: [Quick start](https://scrape.ly/docs/quickstart.md) - Next: [How Actors work](https://scrape.ly/docs/actors.md) - All docs: [Docs index](https://scrape.ly/docs.md), [llms.txt](https://scrape.ly/llms.txt)