---
title: "Errors & deployment · Scrape.ly Docs"
description: "Exceptions, troubleshooting, Dockerfiles, and testing."
url: https://scrape.ly/docs/sdk-python-deploy
markdown: https://scrape.ly/docs/sdk-python-deploy.md
---

# Errors & deployment

> Exceptions, troubleshooting, Dockerfiles, and testing.

This page covers the exceptions the Python SDK actually raises, common failures and their fixes, and how to package, test and deploy a Python Actor on Scrape.ly.

## Errors

### What the `Actor` methods raise

| Exception | Raised by | When |
| --- | --- | --- |
| `scrapely.storage.ApiClientError` | `start`, `call`, `call_task`, `abort`, `metamorph`, `reboot`, `add_webhook`, `set_status_message`, `new_client()` requests | The API returned an error status. It has `status_code` and `body` attributes |
| `TimeoutError` | `call`, `call_task` | `timeout` expired. The child run was aborted |
| `RuntimeError` | `call`, `call_task` | The token can't read the child run (HTTP 403) |
| `RuntimeError` | Most methods | Called before `Actor.init()`, or `init()` found no credentials on the platform |
| `ValueError` | `get_input_or_throw`, `abort`, `open_*`, `push_data`, `charge`, `ProxyConfiguration`, `BrowserConfiguration`, `Webhook` | Missing input or invalid arguments |
| `ChargeUncertainError` | `charge`, `push_data` with a charged event | The platform didn't confirm a charge. It has `operation_id`, `event_name` and `count` |
| `SystemExit` | `exit`, `fail`, `metamorph`, `reboot`, `Actor.main` | Raised on purpose to end the process |

```python
from scrapely import Actor, CallOptions
from scrapely.storage import ApiClientError

try:
    run = await Actor.call("owner~child-actor", {"url": url}, CallOptions(timeout=600))
except ApiClientError as error:
    if error.status_code == 404:
        Actor.log.warning("Actor does not exist")
    else:
        raise
except TimeoutError:
    Actor.log.warning("Child run timed out and was aborted")
else:
    if run.status != "SUCCEEDED":
        Actor.log.warning("Child run ended with %s", run.status)
```

A failed child run isn't an exception, so check `run.status`. `ApiClientError` already retries HTTP 429 three times before it raises. Errors from `Actor.apify_client` come from `scrapely_client`. See the [Python API client](https://scrape.ly/docs/client-python.md).

### Exception hierarchy

The root package also exports a typed hierarchy:

```
ScrapelyClientError
├── InvalidResponseBodyError
└── ScrapelyApiError
    ├── InvalidRequestError    (400)
    ├── UnauthorizedError      (401)
    ├── ForbiddenError         (403)
    ├── NotFoundError          (404)
    ├── ConflictError          (409)
    ├── RateLimitError         (429)
    └── ServerError            (5xx)
```

The `Actor` methods don't raise these. Only `raise_for_status(response, attempt=1, *, method="GET")` raises them, for an `httpx.Response` you got yourself. `ScrapelyApiError` has `response`, `attempt`, `method`, `status_code`, `type`, `message` and `data`.

`ChargeRejectedError` is exported too, but the SDK handles it internally: a rejected charge returns `charged_count=0`.

## Troubleshooting

| Symptom | Cause | Fix |
| --- | --- | --- |
| `ModuleNotFoundError: scrapely` locally | The SDK only ships in Scrape.ly images | Run on the platform, or develop in a `scrapely/actor-python*` container |
| Wrong `scrapely` package installed | `scrapely` is listed in `requirements.txt` | Remove it. The SDK is preinstalled |
| `requires Actor.init()` | An SDK method was called before init | Use `async with Actor:` or call `Actor.init()` |
| `Missing required configuration for secure storage` | Platform run without API URL, token or run ID | Let the platform start the container, or pass `InitOptions(storage=...)` |
| `Input does not exist` | The run has no input | Provide input, or handle `None` from `get_input()` |
| Run fails after `Actor.exit()` | `Actor.exit()` called inside `Actor.main()` | Return from the function instead. See [Lifecycle](https://scrape.ly/docs/sdk-python-lifecycle.md#actormain) |
| `Actor failed: 0` in the log | `Actor.exit()` inside `async with Actor:` | Harmless. The run still exits with code 0 |
| Metamorph target sees old input | `get_input()` reads `INPUT-METAMORPH-<n>` first | Read `await Actor.get_value("INPUT")` |
| `create_proxy_configuration()` returns `None` | No custom proxies and no platform proxy credentials | Enable proxy access, or pass `proxy_urls` |
| Browser ignores your proxy | The Actor uses the [Scrape.ly browser](https://scrape.ly/docs/browser.md), which has its own proxy | Use `BrowserConfiguration` for location, or turn the setting off to use your proxy |
| `use_state()` changes lost | Nothing triggered a save | Call `persist_autosaved_values()`. See [State persistence](https://scrape.ly/docs/sdk-python-events.md#state-persistence) |
| `add_webhook()` returns `None` | Not running on the platform | Run on Scrape.ly, or use the [webhooks API](https://scrape.ly/docs/api-webhooks.md) |
| `charged_count` is `0` | Budget reached, the charge was rejected, or the Actor isn't pay-per-event | Check `event_charge_limit_reached` and stop |
| Output missing | A write wasn't awaited | Await every write |

## Packaging

The SDK is preinstalled. Don't add `scrapely` or `apify` to `requirements.txt`. For details, see [Getting the SDK](https://scrape.ly/docs/sdk-python.md#getting-the-sdk).

### Without a Dockerfile

If your project has `requirements.txt` or `.py` files and no Dockerfile, the build generates one. It uses the `scrapely/actor-python:3.12` image, installs `requirements.txt` and runs `python /actor/main.py`. Put your entrypoint in `main.py` at the project root.

### With a Dockerfile

A typical Python Actor uses this layout:

```dockerfile
FROM scrapely/actor-python:3.14-latest

COPY requirements.txt ./
RUN pip install -r requirements.txt

COPY . ./
CMD ["python", "-m", "my_actor"]
```

Pick a browser image, such as `scrapely/actor-python-playwright:3.14-latest`, for Playwright crawlers. The platform rewrites the tag to the SDK release it currently ships. If you use an `apify/actor-python*` base, the build swaps it for the matching Scrape.ly image, but publishing requires a `scrapely/actor-python*` base. See [Publishing](https://scrape.ly/docs/publishing.md).

A project usually contains:

```
.actor/
  scrapely.json
  input_schema.json
  dataset_schema.json
Dockerfile
requirements.txt
my_actor/
  __init__.py
  __main__.py
  main.py
```

## Testing

On your machine:

- Unit-test pure helpers, such as parsing and URL normalization, with ordinary inputs.
- Run `scrapely validate-schema` to catch a broken input schema before you push.
- `scrapely run` runs the project locally with file-system storage in `./storage`. It needs an environment where `import scrapely` works.

On the platform:

```bash
scrapely push
scrapely call -i '{"startUrls":[{"url":"https://example.com"}]}' -o
scrapely runs log <runId>
scrapely datasets get-items <datasetId> --format json --limit 20
```

Start with a small input. Test charging, webhooks, abort, reboot and metamorph with real runs, because they don't work off the platform.

Before you ship a larger crawl:

1. Keep `scrapely` and `apify` out of `requirements.txt`.
2. Validate the input before expensive work.
3. Await every dataset, key-value store and queue write.
4. Persist crawl state with `use_state()` and `persist_autosaved_values()` if a migration would lose progress.
5. Handle `Event.ABORTING` if you need to clean up.
6. Keep tokens and proxy credentials out of source and logs.

## Production template

```python
import asyncio

from crawlee.crawlers import BeautifulSoupCrawler, BeautifulSoupCrawlingContext
from scrapely import Actor, Event


async def main() -> None:
    async with Actor:
        async def on_aborting() -> None:
            await asyncio.sleep(1)
            await Actor.exit()

        Actor.on(Event.ABORTING, on_aborting)

        actor_input = await Actor.get_input() or {}
        start_urls = [item["url"] for item in actor_input.get("startUrls", [])]
        if not start_urls:
            await Actor.fail("startUrls is required")

        proxy_configuration = await Actor.create_proxy_configuration(actor_input.get("proxyConfiguration"))
        crawler = BeautifulSoupCrawler(
            proxy_configuration=proxy_configuration,
            max_requests_per_crawl=actor_input.get("maxRequestsPerCrawl", 100),
        )

        @crawler.router.default_handler
        async def handler(context: BeautifulSoupCrawlingContext) -> None:
            title = context.soup.title.string.strip() if context.soup.title else ""
            await context.push_data({"url": context.request.url, "title": title})
            await context.enqueue_links()

        await crawler.run(start_urls)


if __name__ == "__main__":
    asyncio.run(main())
```

`context.push_data()` writes to the default dataset through Crawlee. Use `Actor.push_data(..., charged_event_name=...)` when each item should be charged.

---

- Previous: [Environment & proxy](https://scrape.ly/docs/sdk-python-environment.md)
- Next: [API reference](https://scrape.ly/docs/sdk-python-reference.md)
- All docs: [Docs index](https://scrape.ly/docs.md), [llms.txt](https://scrape.ly/llms.txt)