---
title: "Python client · Scrape.ly Docs"
description: "Sync and async Python client for running Actors and reading results."
url: https://scrape.ly/docs/client-python
markdown: https://scrape.ly/docs/client-python.md
---

# Python client

> Sync and async Python client for running Actors and reading results.

`scrapely-client-python` is the Python client for the [Scrape.ly API](https://scrape.ly/docs/api-overview.md), with a synchronous and an asynchronous interface. Use it from your own scripts, backends, notebooks, or data pipelines to start Actors, wait for runs, and read and write storage. It is not for writing Actors: code that runs inside an Actor should use the [Python SDK](https://scrape.ly/docs/sdk-python.md).

The source code is on GitHub: [scrapely-app/scrapely-client-python](https://github.com/scrapely-app/scrapely-client-python).

The client is a drop-in replacement for `apify-client`. Method names, arguments, and return models match, so most existing code works after you change the import and the API URL.

## Install

The client requires Python 3.11 or newer.

```bash
pip install scrapely-client-python
```

The distribution is called `scrapely-client-python`, but the import name is `scrapely_client`:

```python
from scrapely_client import ScrapelyClient, ScrapelyClientAsync
```

`ApifyClient` and `ApifyClientAsync` are exported as aliases. For Brotli response compression, install the extra: `pip install "scrapely-client-python[brotli]"`.

## Create a client

```python
import os
from scrapely_client import ScrapelyClient

client = ScrapelyClient(
    os.environ['SCRAPELY_TOKEN'],
    api_url='https://api.scrape.ly',
)
```

> [!WARNING]
> Always set `api_url` to `https://api.scrape.ly`. The current release defaults to a non-production URL, so a client without `api_url` does not talk to your Scrape.ly account.

Create a token in the dashboard under **Settings → API & Integrations**. The client does not read the token or the API URL from environment variables; pass them explicitly. The token is sent as `Authorization: Bearer <token>`.

| Argument | Default | Description |
| --- | --- | --- |
| `token` | `None` | API token, the first positional argument. Without it, only public endpoints work. |
| `api_url` | non-production URL | API origin, without `/v2`. Set to `https://api.scrape.ly`. |
| `api_public_url` | non-production URL | Origin used when the client builds public storage URLs. Set to `https://api.scrape.ly`. |
| `max_retries` | `4` | Retries for `429`, `5xx`, and retryable network errors. |
| `min_delay_between_retries` | `timedelta(milliseconds=500)` | Initial retry delay. It grows exponentially with jitter. |
| `timeout_short` / `timeout_medium` / `timeout_long` | 5 s / 30 s / 360 s | Timeouts used by the `'short'`, `'medium'`, and `'long'` tiers. |
| `timeout_max` | 360 s | Upper limit for a single attempt. |
| `headers` | `None` | Extra headers sent with every request. |
| `compression` | `'gzip'` | Request body compression: `'gzip'`, or `'brotli'` with the extra installed. |

Every argument after `token` is keyword-only.

## Sync and async clients

`ScrapelyClient` blocks; `ScrapelyClientAsync` has the same methods as coroutines. Resource accessors such as `client.actor(...)` are plain calls in both; only the API calls are awaited.

```python
import asyncio
from scrapely_client import ScrapelyClientAsync

async def main() -> None:
    client = ScrapelyClientAsync('MY-TOKEN', api_url='https://api.scrape.ly')
    run = await client.actor('username/my-actor').call(run_input={'url': 'https://example.com'})
    async for item in client.dataset(run.default_dataset_id).iterate_items():
        print(item)

asyncio.run(main())
```

## Run an Actor

`actor(id).call()` starts a run and waits until it finishes. Pass an Actor ID or a `username/actor-name` full name.

```python
run = client.actor('username/my-actor').call(run_input={'url': 'https://example.com'})

print(run.status)  # SUCCEEDED, FAILED, ABORTED, or TIMED-OUT

for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)
```

Return values are Pydantic models with snake_case attributes: `run.id`, `run.act_id`, `run.status`, `run.default_dataset_id`, `run.default_key_value_store_id`, and so on. `call()` returns the run for every final status; it does not raise when the run fails, so check `run.status`. It returns `None` only if the run can no longer be found.

On runs of someone else's Actor, `run.build_id`, `run.build_number`, and `run.options.build` are `None`: build info is owner only, and the run uses the [User Build](https://scrape.ly/docs/publishing.md#user-build). Older client versions raise a validation error on these runs, so keep the client up to date.

To start a run without waiting, use `start()`, then wait later:

```python
from datetime import timedelta

run = client.actor('username/my-actor').start(
    run_input={'url': 'https://example.com'},
    memory_mbytes=1024,
    run_timeout=timedelta(minutes=5),
)

finished = client.run(run.id).wait_for_finish(wait_duration=timedelta(minutes=2))
```

`wait_for_finish()` polls until the run reaches a final status or `wait_duration` elapses. Without `wait_duration` it waits indefinitely.

Run arguments for `start()` and `call()`:

| Argument | Description |
| --- | --- |
| `run_input` | Actor input, usually a dict. |
| `content_type` | Content type of `run_input` when it is not JSON. |
| `build` | Build tag or number. Defaults to the Actor's default build. Ignored on someone else's Actor, which always runs the [User Build](https://scrape.ly/docs/publishing.md#user-build). |
| `memory_mbytes` | Memory in MB. |
| `run_timeout` | Run timeout as a `timedelta`. |
| `max_items` | Cap on billed dataset items for Actors that charge per result item. |
| `max_total_charge_usd` | Cap on the total charge in USD for pay-per-event Actors, as a `Decimal`. |
| `restart_on_error` | Restart the run if it fails. |
| `webhooks` | Ad-hoc webhooks for this run only. |
| `wait_for_finish` | `start()` only. Seconds the API holds the request open (up to 60; the server stops waiting at 45). |
| `wait_duration` | `call()` only. Maximum time to wait. Omit to wait indefinitely. |
| `logger` | `call()` only. Where to redirect the run log: `'default'` (the default), a `logging.Logger`, or `None` to disable. |

The API returns a run's log as a snapshot rather than a live stream, so the redirected log in `call()` may be incomplete. Fetch the full log after the run with `client.run(run.id).log().get()`.

See [Run Actors](https://scrape.ly/docs/run-actors.md) for how runs, memory, and timeouts work on the platform.

### Other Actor methods

| Method | Description |
| --- | --- |
| `actor(id).get()` | Actor details, or `None` if it does not exist. |
| `actor(id).update(...)` / `.delete()` | Change or remove your Actor. Set the [User Build](https://scrape.ly/docs/publishing.md#user-build) with `update(default_non_owner_build='0.1.4')`; `''` clears it. |
| `actor(id).build(version_number=..., tag=..., use_cache=..., wait_for_finish=...)` | Start a build. |
| `actor(id).default_build()` | Returns a build client for the default build. Owner only; for other Actors read the input schema from `get()` (`defaultBuildActorDefinition`). |
| `actor(id).last_run(status=...)` | Run client for the latest run. |
| `actor(id).runs()` / `.builds()` / `.versions()` / `.webhooks()` | Related collections. Builds and versions are owner only. |
| `actors().list(my=..., limit=..., offset=..., desc=...)` | Actors you created or used. |

## Runs

| Method | Description |
| --- | --- |
| `run(id).get()` | Run details, or `None`. |
| `run(id).wait_for_finish(wait_duration=...)` | Poll until the run finishes. |
| `run(id).abort(gracefully=True)` | Abort the run. |
| `run(id).resurrect(...)` | Restart a finished run with the same storages. |
| `run(id).metamorph(target_actor_id=..., run_input=...)` | Replace the running Actor with another one. |
| `run(id).reboot()` | Restart the run's container. |
| `run(id).update(status_message=..., is_status_message_terminal=...)` | Update the run's status message. |
| `run(id).charge(event_name, count=..., idempotency_key=...)` | Charge a pay-per-event event. |
| `run(id).delete()` | Delete the run. |
| `run(id).dataset()` / `.key_value_store()` / `.request_queue()` | Clients for the run's default storages. |
| `run(id).log()` | Log client for the run. |
| `runs().list(status=..., limit=..., offset=..., desc=...)` | All your runs. |

## Datasets

```python
dataset = client.dataset('DATASET_ID')

page = dataset.list_items(limit=100, fields=['title', 'url'])
print(page.total, len(page.items))

csv_bytes = dataset.get_items_as_bytes(item_format='csv', bom=True)
```

| Method | Description |
| --- | --- |
| `list_items(...)` | One page: `items`, `total`, `offset`, `count`, `limit`, `desc`. Arguments: `offset`, `limit`, `fields`, `omit`, `unwind`, `flatten`, `desc`, `clean`, `skip_empty`, `skip_hidden`, `view`. |
| `iterate_items(..., chunk_size=...)` | Iterator over every item, fetched page by page. |
| `get_items_as_bytes(item_format=...)` | Items as `bytes`. Formats: `json`, `jsonl`, `csv`, `xlsx`, `xml`, `rss`, `html`. Extra arguments: `bom`, `delimiter`, `skip_header_row`, `xml_root`, `xml_row`. |
| `stream_items(item_format=...)` | Context manager yielding the raw HTTP response, for large exports. |
| `push_items(items)` | Append a dict, a list of dicts, or a JSON string. |
| `get_statistics()` | Dataset statistics. |
| `create_items_public_url(expires_in=...)` | Signed URL for the items. |
| `datasets().list(unnamed=...)` / `.get_or_create(name=...)` | Your datasets, or a named dataset. |

See [Results](https://scrape.ly/docs/results.md) and [Storage](https://scrape.ly/docs/storage.md) for the data model.

## Key-value stores

```python
store = client.key_value_store('STORE_ID')

store.set_record('config', {'depth': 2})

record = store.get_record('OUTPUT')
if record:
    print(record['content_type'], record['value'])
```

| Method | Description |
| --- | --- |
| `get_record(key)` | Dict with `key`, `value`, and `content_type`, or `None` if the key does not exist. JSON is parsed and text is decoded. |
| `get_record_as_bytes(key)` / `stream_record(key)` | Raw bytes, or a streamed response for large records. |
| `set_record(key, value, content_type=...)` | Store a record. Dicts and lists are stored as JSON by default. |
| `record_exists(key)` | `True` or `False`. |
| `delete_record(key)` | Delete a record. |
| `list_keys(limit=..., exclusive_start_key=..., prefix=..., collection=...)` | One page of keys with `is_truncated` and `next_exclusive_start_key`. |
| `iterate_keys(...)` | Iterator over every key. |
| `get_record_public_url(key)` / `create_keys_public_url()` | Signed public URLs. |
| `key_value_stores().list()` / `.get_or_create(name=...)` | Your stores, or a named store. |

## Request queues

```python
queue = client.request_queue('QUEUE_ID')

queue.add_request({'url': 'https://example.com', 'uniqueKey': 'https://example.com'})
head = queue.list_head(limit=10)
```

Available methods: `add_request`, `batch_add_requests`, `get_request`, `update_request`, `delete_request`, `batch_delete_requests`, `list_head`, `list_and_lock_head`, `prolong_request_lock`, `delete_request_lock`, `unlock_requests`, `list_requests`, `iterate_requests`, plus `get`, `update`, and `delete`. `request_queues().get_or_create(name=...)` returns a named queue.

## Tasks

```python
task = client.tasks().create(
    actor_id='username/my-actor',
    name='daily-crawl',
    task_input={'url': 'https://example.com'},
)

run = client.task(task.id).call()
```

Task names are limited to 30 characters.

| Method | Description |
| --- | --- |
| `task(id).start(task_input=..., ...)` / `.call(...)` | Run the task. `task_input` overrides the stored input. Arguments: `build` (your own Actors only), `max_items`, `memory_mbytes`, `run_timeout`, `restart_on_error`, `webhooks`, and `wait_for_finish` or `wait_duration`. |
| `task(id).get_input()` / `.update_input(task_input)` | Read or replace the stored input. |
| `task(id).get()` / `.update(...)` / `.delete()` | Task details. |
| `task(id).publish(...)` / `.unpublish()` | Publish the task as a public page or take it down. |
| `task(id).last_run()` / `.runs()` / `.webhooks()` | Related runs and webhooks. |
| `tasks().list()` | Your tasks. |

## Schedules

```python
client.schedules().create(
    name='nightly',
    cron_expression='0 3 * * *',
    timezone='UTC',
    is_enabled=True,
    is_exclusive=False,
    actions=[{'type': 'RUN_ACTOR', 'actorId': 'ACTOR_ID'}],
)
```

`cron_expression`, `is_enabled`, and `is_exclusive` are required arguments, and the API requires at least one action. Actions are `RUN_ACTOR` (with `actorId`) or `RUN_ACTOR_TASK` (with `actorTaskId`). Other methods: `schedules().list()`, `schedule(id).get()`, `.update(...)`, `.delete()`, and `.get_log()`. See [Schedules API](https://scrape.ly/docs/api-schedules.md).

## Automations

```python
automation = client.automations().create({
    'title': 'Morning digest',
    'trigger': {'kind': 'schedule', 'cron': '0 7 * * 1-5', 'timezone': 'Europe/Berlin'},
    'steps': [{'kind': 'savedInput', 'savedInputId': 'SAVED_INPUT_ID', 'inputOverride': {'maxItems': 20}}],
    'filters': {'onlyNewItems': {'keyFields': ['url']}},
    'ai': {'mode': 'summarize', 'prompt': 'The five most interesting new items, with links.'},
    'deliveries': [{'channel': 'email'}],
})

run = client.automation(automation['id']).run()
```

| Method | Description |
| --- | --- |
| `automations().list(actor_id=..., saved_input_id=..., search=..., limit=..., offset=..., desc=...)` | Your automations, plain schedules included. `actor_id` and `saved_input_id` return the automations that use them. |
| `automations().create(automation)` | Create an automation from a dictionary with the body of [Create automation](https://scrape.ly/docs/api-automations.md#create-automation). |
| `automation(id).get()` / `.update(fields)` / `.delete()` | Read, partly update, or delete it. `update` sends the fields as given; `{'ai': None}` removes the AI step. |
| `automation(id).run()` | Run it now. Returns the automation run. |
| `automation(id).list_runs(limit=..., offset=...)` / `.get_run(run_id)` | Run history and one run, with each step's run, the filter result, `aiOutput`, `aiCostUsd`, and delivery statuses. |
| `automation(id).duplicate(title=..., name=..., is_enabled=...)` | Copy it. The copy is paused unless `is_enabled` is `True`. |

The methods return the API's dictionaries with camelCase keys, and the async client has the same methods. `automations()` is Scrape.ly-only and comes with version 1.0.6 of `scrapely-client-python`.

## Webhooks

```python
client.webhooks().create(
    event_types=['ACTOR.RUN.SUCCEEDED', 'ACTOR.RUN.FAILED'],
    actor_id='ACTOR_ID',
    request_url='https://example.com/hooks/scrapely',
)
```

Other methods: `webhooks().list()`, `webhook(id).get()`, `.update(...)`, `.delete()`, `.test()`, `.dispatches()`, and `webhook_dispatches().list()`. See [Webhooks API](https://scrape.ly/docs/api-webhooks.md).

## Builds, account, and Store

Builds are owner only, also for public Actors; other users get `403` on an Actor's builds and `404` on a build by ID.

| Method | Description |
| --- | --- |
| `build(id).get()` / `.wait_for_finish(wait_duration=...)` | Build details. |
| `build(id).abort()` / `.delete()` / `.log()` | Manage a build and read its log. |
| `user().get()` | The account that owns the token. |
| `user().monthly_usage()` / `.limits()` | Usage and limits for the current month. |
| `store().list(search=..., limit=..., offset=..., sort_by=..., category=..., username=..., pricing_model=...)` | Search the Store. The API defaults `limit` to 20 and caps it at 100. |
| `store().iterate(...)` | Iterator over every matching Store Actor. |

## Logs

```python
text = client.run('RUN_ID').log().get()

with client.run('RUN_ID').log().stream() as response:
    if response:
        for chunk in response.iter_bytes():
            print(chunk.decode(), end='')
```

`get()` returns the log as a string, or `None`. With the async client, use `async with` and `aiter_bytes()`. The API sends the log as it is at request time and then closes the connection, so to follow a running Actor, poll `get()`.

`run(id).get_streamed_log(to_logger)` returns a `StreamedLog` you can use as a context manager (or call `start()` and `stop()`) to redirect a run's log into a `logging.Logger`; it has the same snapshot limitation. The client logs under the `scrapely_client` logger name.

## Pagination

Every collection and storage listing has two forms:

- `list(...)`, `list_items(...)`, `list_keys(...)` return a single page with its metadata.
- `iterate(...)`, `iterate_items(...)`, `iterate_keys(...)`, `iterate_requests(...)` return an iterator that fetches pages as you go. With the async client, use `async for`.

```python
for run in client.runs().iterate(desc=True):
    print(run.id, run.status)
```

`limit` and `offset` bound the overall range; `chunk_size` sets how many items each request fetches.

## Timeouts, retries, and errors

Every API method takes a keyword-only `timeout`: a `timedelta`, one of `'short'`, `'medium'`, `'long'`, or `'no_timeout'`. Each retry doubles the timeout, up to `timeout_max`. `call()` and `wait_for_finish()` default to `'no_timeout'`.

The client retries `429`, `5xx`, and retryable network errors up to `max_retries` times with exponential backoff and jitter. Other `4xx` responses fail immediately.

Failed calls raise `ScrapelyApiError` from `scrapely_client.errors` (also available as `ApifyApiError`). The client raises a subclass based on the HTTP status:

| Exception | Status |
| --- | --- |
| `InvalidRequestError` | 400 |
| `UnauthorizedError` | 401 |
| `ForbiddenError` | 403 |
| `NotFoundError` | 404 |
| `ConflictError` | 409 |
| `RateLimitError` | 429 |
| `ServerError` | 5xx |

```python
from scrapely_client.errors import NotFoundError, ScrapelyApiError

try:
    client.actor('username/missing').start()
except NotFoundError:
    print('No such Actor')
except ScrapelyApiError as err:
    print(err.status_code, err.type, err.message, err.attempt)
```

`get()` methods return `None` on `404` instead of raising. See [API overview](https://scrape.ly/docs/api-overview.md) for error types.

## Migrating from apify-client

- Install `scrapely-client-python` and import from `scrapely_client`. `ApifyClient`, `ApifyClientAsync`, and `ApifyApiError` still work as aliases.
- Pass `api_url='https://api.scrape.ly'` (and `api_public_url` if you build public URLs).
- Use a Scrape.ly token. Apify tokens do not work.
- `task(id).publish()` and `.unpublish()` are Scrape.ly additions.
- Endpoints that Scrape.ly does not implement return an error. The [API reference](https://scrape.ly/docs/api.md) lists what is available.

---

- Previous: [JavaScript client](https://scrape.ly/docs/client-js.md)
- Next: [MCP server](https://scrape.ly/docs/mcp.md)
- All docs: [Docs index](https://scrape.ly/docs.md), [llms.txt](https://scrape.ly/llms.txt)