Python SDK

The Python SDK is the runtime library for Python Actors on Scrape.ly. It gives you the Actor lifecycle API, input and storage access, platform events, Actor-to-Actor calls, webhooks, pay-per-event charging, the platform proxy and the opt-in Scrape.ly browser. It is built on Crawlee for Python, and its API mirrors the Apify Python SDK, so most Apify Actors run with few or no changes.

The source code is on GitHub: scrapely-app/scrapely-python.

Getting the SDK

You don't install the SDK for platform runs. It comes preinstalled in the Scrape.ly Python base images, and every Python build on Scrape.ly runs on one of those images. The SDK is published on PyPI as scrapely-python and imported as scrapely.

The base images put the SDK in /opt/scrapely/site-packages, outside your virtual environment. They add that directory to PYTHONPATH and set SCRAPELY_IS_AT_HOME=1. The standalone API client, scrapely-client-python (import name scrapely_client), is preinstalled the same way.

You get the SDK in one of these three ways:

Every Python build also runs a final step. That step uninstalls apify, pins crawlee>=1.8,<2, reinstalls the SDK's own dependencies and runs pip check. Any conflicts pip check reports are logged as warnings and don't fail the build. So if your requirements.txt pulls in an incompatible Crawlee or the apify package, the build corrects it.

Base images

Each image family below is available for Python 3.11, 3.12, 3.13 and 3.14. Tags look like scrapely/actor-python-playwright:3.14-latest. At build time, the platform rewrites the part after the Python version to the SDK release it currently ships.

The SDK requires Python 3.11 or newer. To publish an Actor, its Dockerfile must use a scrapely/actor-python* base on Python 3.11 or newer, or the project must have no Dockerfile. See Publishing.

requirements.txt

List only the packages your code needs, for example crawlee[beautifulsoup] or crawlee[playwright]. Leave out scrapely-python, scrapely and apify.

Local runs

scrapely run runs your project on your machine with .venv/bin/python, falling back to python3. It looks for main.py, src/main.py or the entrypoint you pass with --entrypoint. Outside the platform, the SDK uses Crawlee's file-system storage in ./storage, and it launches a local browser instead of the remote one. Charging, platform events and the platform proxy aren't available.

To import scrapely locally, install the SDK into your virtual environment with pip install scrapely-python, but keep it out of requirements.txt. To test against the platform itself, use scrapely push and scrapely call, or develop inside a scrapely/actor-python* container.

Imports

from scrapely import Actor, Event
from crawlee.crawlers import BeautifulSoupCrawler, PlaywrightCrawler

Crawler classes come from crawlee.crawlers. The SDK root re-exports Crawlee's Dataset, KeyValueStore, RequestQueue, Request, Event and EventListener. It also exports Scrape.ly's own Configuration, ProxyConfiguration, option dataclasses, charging helpers, webhooks, BrowserConfiguration, ScrapelyRequestList and an exception hierarchy. See the API reference for the full list.

Importing scrapely patches Crawlee's browser pool, Playwright, pyppeteer, and Camoufox. When the Actor has Use Scrapely browser turned on, platform launches connect to the Scrape.ly browser; with it off (the default) they launch in the container. See Remote browser.

The base images also ship an apify module that re-exports scrapely. That means from apify import Actor resolves to this SDK.

Async model

The SDK is async. Most Actor methods are coroutines, so you must await them inside an event loop.

These members are synchronous: Actor.on(), Actor.off(), Actor.get_env(), Actor.get_config(), Actor.is_at_home(), Actor.is_standby(), Actor.get_remaining_time(), Actor.get_charging_manager(), Actor.new_client() and Actor.get_default_instance(). The log, input, exit_code, status_message, is_paying, configuration, event_manager and apify_client properties are synchronous too.

Minimal Actor

Use async with Actor::

import asyncio

from scrapely import Actor


async def main() -> None:
    async with Actor:
        actor_input = await Actor.get_input() or {}
        await Actor.push_data({"url": actor_input.get("url"), "ok": True})


if __name__ == "__main__":
    asyncio.run(main())

Entering the block calls Actor.init(). Leaving it flushes pending charges, closes events and closes storages. Actor.main(main) is an equivalent entrypoint that also calls sys.exit() with the run's exit code. See Lifecycle for the differences.

How an Actor runs

The platform starts your container with the run context in environment variables. These include the run ID, a scoped API token, the API URL, the default storage IDs, the input, the timeout, the events WebSocket URL, the Scrape.ly browser URL and, when enabled, platform proxy credentials. The SDK reads them during Actor.init().

1. Actor.init() connects storage to the Scrape.ly API, connects the events WebSocket and loads pricing information for pay-per-event Actors.
2. Your code reads input with Actor.get_input().
3. Your crawler or code writes results with Actor.push_data(), Actor.set_value() or an opened storage.
4. The platform sends systemInfo, persistState, migrating and aborting events over the WebSocket.
5. On exit, the SDK flushes pending charges, closes the event manager and closes storages.

Always await writes. If you don't, the process can end before a dataset or key-value store write reaches the API.

Where to go next

To call Actors or read storages from outside an Actor, use the standalone Python API client.