Lifecycle

Every Actor initializes the SDK, does its work and shuts down. Shutdown flushes pending charges, closes the event connection and closes storages. This page covers the three ways to manage that: the async with Actor: context manager, Actor.main() and manual init()/exit(). It also covers how exit codes, status messages and signals behave.

Context manager

async with Actor: is the recommended form:

import asyncio

from scrapely import Actor


async def main() -> None:
    async with Actor:
        actor_input = await Actor.get_input() or {}
        await Actor.push_data({"received": actor_input})


if __name__ == "__main__":
    asyncio.run(main())

Entering the block calls Actor.init(), and leaving it shuts the SDK down. If the block raises, the SDK sets Actor.exit_code to 1, logs Actor failed: <error>, shuts down and re-raises. The process then exits with the uncaught exception. The context manager never calls sys.exit() itself.

Instance form

To pass options, instantiate Actor:

from datetime import timedelta

async with Actor(configure_logging=False, cleanup_timeout=timedelta(seconds=10)):
    ...

exit_code and status_message only set the class attributes. They don't end the process with that code and don't update the run. To do that, call Actor.exit() or Actor.set_status_message().

Actor.main()

Actor.main(user_func) runs async with Actor: around your coroutine function, then always calls sys.exit(Actor.exit_code):

import asyncio

from scrapely import Actor


async def main():
    actor_input = await Actor.get_input_or_throw()
    await Actor.push_data({"url": actor_input["url"], "ok": True})


if __name__ == "__main__":
    asyncio.run(Actor.main(main))

If main raises an exception, the SDK logs it, the exit code is 1 and the exception is re-raised before sys.exit(1). Don't nest async with Actor: inside the callback. The nested init() call is a no-op, but the inner block shuts the SDK down early.

Manual init and exit

Use Actor.init() and Actor.exit() directly when you need full control:

import asyncio

from scrapely import Actor


async def main():
    await Actor.init()
    try:
        actor_input = await Actor.get_input_or_throw()
        await Actor.push_data({"received": actor_input})
    except Exception as error:
        await Actor.fail(status_message=str(error))
    await Actor.exit(status_message="Completed")


if __name__ == "__main__":
    asyncio.run(main())

Actor.init() is idempotent. If initialization fails, the SDK rolls back partial state, so you can call init() again. Methods that need the SDK, such as on(), get_charging_manager() and create_proxy_configuration(), raise RuntimeError when called before init().

Actor.exit() and Actor.fail()

await Actor.exit(message_or_options=None, options=None, *, status_message=None, exit_code=None)
await Actor.fail(message_or_options=None, options=None, *, status_message=None, exit_code=None, exception=None)

The first argument can be a message string or an ExitOptions. You can also pass status_message and exit_code as keyword arguments. fail() also takes exception, and logs that exception's traceback.

from scrapely import Actor, ExitOptions

await Actor.exit("Finished")
await Actor.exit(status_message="Finished", exit_code=0)
await Actor.exit(ExitOptions(status_message="Finished", exit=False))
await Actor.fail("Validation failed")
await Actor.fail(status_message="Upstream down", exit_code=2)

try:
    ...
except Exception as e:
    await Actor.fail(status_message="Crashed", exception=e)

ExitOptions:

exit() logs the status message, shuts down, then calls sys.exit(exit_code) unless exit=False. fail() logs at error level and defaults the exit code to 1.

When you call Actor.exit() inside async with Actor:, the process still exits with your code. However, the context manager also logs a spurious Actor failed: 0 line.

Signals

Actor.init() installs handlers for SIGINT and SIGTERM. On either signal, the SDK flushes pending pay-per-event charges and closes events and storages. It allows about 5 seconds for this, then ends the process with os._exit(0). Code after the signal doesn't run, so persist what you need on the aborting or persistState events. See Events & state.

Initialization

from scrapely import Actor, InitOptions

await Actor.init(InitOptions(configuration={"persist_state_interval_millis": 30000}))

InitOptions:

The SDK doesn't install automatic aborting or migrating handlers. If you want to react to those events, register listeners yourself. See Events & state.

Storage selection

init() picks storage in this order:

1. If InitOptions.storage is a mapping {"api_url", "token", "run_id"}, use Scrape.ly API storage for that run. Its values also become the configuration used for events and charging.
2. If InitOptions.storage is a Crawlee StorageClient, use it. Any other value raises TypeError.
3. If the configuration has an API URL, a token and a run ID, use API storage. This is the normal platform case.
4. If not running on the platform, use Crawlee file-system storage in ./storage, or in CRAWLEE_STORAGE_DIR.
5. Otherwise, raise RuntimeError("Missing required configuration for secure storage…").

All of these are wrapped in a SmartScrapelyStorageClient. Off the platform, it still opens storages locally unless you pass force_cloud=True, even when you provide API credentials. On the platform, the SDK disables Crawlee's purge-on-start, so the run keeps its existing storages. See Storage backends.

Event manager selection

When ACTOR_EVENTS_WEBSOCKET_URL is set, init() connects a ScrapelyEventManager to the platform. Otherwise it uses Crawlee's LocalEventManager, which emits systemInfo every second. Both emit persistState every persist_state_interval_millis, which defaults to 60000.

Configuration

Actor.configuration (or Actor.get_config()) returns the global Configuration. It extends the Crawlee configuration with platform fields that are loaded from environment variables. See Environment.

config = Actor.configuration
config.get("actor_run_id")
config.set("persist_state_interval_millis", 30000)

Configuration.get_global_config() returns the same object, and Configuration.reset_global_state() clears it.