---
title: "API overview · Scrape.ly Docs"
description: "Base URL, authentication, token types, errors, rate limits, size limits, and API clients."
url: https://scrape.ly/docs/api-overview
markdown: https://scrape.ly/docs/api-overview.md
---

# API overview

> Base URL, authentication, token types, errors, rate limits, size limits, and API clients.

The scrape.ly API is a REST API. Every public endpoint in these docs is versioned under `/v2`.

**Base URL**

```
https://api.scrape.ly
```

Most requests and responses are JSON (UTF-8). The exceptions are noted on each page: dataset items, key-value store records, Actor pictures, and run and build logs.

```api-test
{ "method": "GET", "path": "/v2/users/me" }
```

## Authentication

Create a token in the dashboard under **Settings → API & Integrations**. Send it on every request in the `Authorization` header:

```bash
curl -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  https://api.scrape.ly/v2/users/me
```

The `/v2` API does not read tokens from the query string. A `?token=` parameter is ignored, so the request is treated as unauthenticated. The one exception is the CI build webhook URL from [`GET /v2/actors/{actorId}/ci/webhook-url`](https://scrape.ly/docs/api-actors.md#ci-webhooks). That URL carries a per-Actor CI secret, not your API key, and it is only accepted by `POST /v2/actors/{actorId}/builds`.

Never commit tokens to source control. Resetting your API key invalidates the old value immediately.

You can create several API keys and limit each one with scopes, resource restrictions, an IP allowlist, and spending caps. See [API keys & permissions](https://scrape.ly/docs/api-keys.md).

A missing token returns `401` with `error.type` `token-not-provided`. An unknown, expired, or revoked token returns `401` with `token-not-valid`.

### When authentication is required

- **Required** to list or change anything you own: Actors, runs, tasks, storages, schedules, and webhooks.
- **Required** for every `/v2/users/me` endpoint.
- **Optional** for public Actor metadata (`GET /v2/actors/{actorId}`), public Actor pictures, public example tasks, and the Store listing.

### Token types

| Token | Prefix | Use |
| --- | --- | --- |
| User API key | `scrapely_` | Full account access from the dashboard, the CLI, or your backend. |
| Additional API key | `scrapely_sk_` | Full access or scoped, created under **Settings → API keys**. See [API keys & permissions](https://scrape.ly/docs/api-keys.md). |
| OAuth access token | `oat_` | Issued through the OAuth flow under `/v2/oauth/*`, for example when you connect an MCP client to the [hosted MCP server](https://scrape.ly/docs/mcp.md). It acts for your account like an API key, but it cannot view or reset API keys. |
| Scoped run token | `st_` | Issued to a running Actor. Its permissions follow the run's permission level. |

**Scoped run tokens** have limits:

- `FULL_PERMISSIONS`: the run can call most of the API on your behalf, including starting other Actors. Account endpoints (`/v2/users/me/*`, API keys) still need a user API key.
- `LIMITED_PERMISSIONS`: the run can use datasets, key-value stores, and request queues, read Actors, start other Actors and tasks (always with limited permissions), and act on its own run and the child runs it started (read, update, abort, metamorph, reboot, resurrect, charge). Every other endpoint returns `403 insufficient-permissions`.

Listing all your runs (`GET /v2/actor-runs`) and creating Actors (`POST /v2/actors`) reject limited scoped tokens. They accept a user API key, an OAuth token, or a full-permission scoped token.

## Response envelope

Most JSON endpoints wrap the payload in `data`:

```json
{
  "data": {
    "id": "eLN5rTB8sYTBWdfzx",
    "name": "my-actor"
  }
}
```

List endpoints use:

```json
{
  "data": {
    "total": 2560,
    "offset": 0,
    "limit": 1000,
    "count": 1000,
    "desc": false,
    "items": []
  }
}
```

**Not wrapped in `data`:** [dataset items](https://scrape.ly/docs/api-datasets.md), [key-value store records](https://scrape.ly/docs/api-key-value-stores.md), [run logs](https://scrape.ly/docs/api-runs.md#get-run-log), [build logs](https://scrape.ly/docs/api-actors.md#get-build-log), run and task **input** JSON, synchronous run output records, Actor pictures, and OpenAPI documents.

Successful deletes usually return `204 No Content` with an empty body. Creates usually return `201` with a `Location` header.

## Pagination

List endpoints accept:

| Parameter | Description |
| --- | --- |
| `limit` | Maximum number of items to return. Capped at `1000`. `0` or no value uses the endpoint default. |
| `offset` | Number of items to skip from the start. |
| `desc` | `desc=1` or `desc=true` returns newest first, where supported. |

A negative `limit` or `offset` returns `400 invalid-input`. Most defaults are `1000`; the Store listing and per-Actor task list default to `20`. Key-value store key listings paginate with `limit` and `exclusiveStartKey` instead. See [Key-value stores](https://scrape.ly/docs/api-key-value-stores.md).

## Errors

Errors use standard HTTP status codes. The body holds an `error` object with `type` and `message`, and sometimes `data` with extra details:

```json
{
  "error": {
    "type": "record-not-found",
    "message": "Store was not found."
  }
}
```

| Status | Type | Meaning |
| --- | --- | --- |
| 400 | `invalid-request` / `invalid-input` / `invalid-value` | Malformed body or parameter. |
| 400 | `invalid-json` | The request body is not valid JSON. |
| 400 | `invalid-memory` / `invalid-timeout` | Memory is not a power of two within your limits, or the timeout is not a positive integer. |
| 400 | `limit-exceeded` | The run or build would exceed your total memory limit. |
| 400 | `unsupported-pricing` | The Actor uses a pricing model that can no longer be run. |
| 401 | `token-not-provided` | No `Authorization: Bearer` header. |
| 401 | `token-not-valid` | The token is unknown, expired, or revoked. |
| 401 | `token-expired` | An [API key](https://scrape.ly/docs/api-keys.md) has passed its expiry date. |
| 402 | `api-key-daily-cap-reached` | The [API key's](https://scrape.ly/docs/api-keys.md#optional-limits) daily charge limit is used up, so the run can't start. |
| 402 | `insufficient-balance` | A balance-gated action (such as a storage write or build) needs a positive balance. |
| 402 | `payment-required` | A run cannot reserve enough credits. |
| 402 | `insufficient-funds` | Your effective balance is used up, for a [standby](https://scrape.ly/docs/api-standby.md) request. |
| 403 | `forbidden` / `insufficient-permissions` | Authenticated but not allowed. For a scoped [API key](https://scrape.ly/docs/api-keys.md), the key's scopes don't cover the request. |
| 403 | `api-key-ip-not-allowed` | The request came from an address outside the [API key's IP allowlist](https://scrape.ly/docs/api-keys.md#optional-limits). |
| 403 | `full-permission-actor-not-approved` | The Actor needs full permissions and you have not approved it yet. `error.data.approvalUrl` links to the approval page. See [Permission approvals](https://scrape.ly/docs/api-actors.md#permission-approvals). |
| 404 | `*-not-found` | The resource does not exist (for example `actor-not-found`, `run-not-found`). |
| 404 | `not-found` | No route matches the method and path. |
| 408 | `run-timeout-exceeded` / `request-timeout` | A synchronous run did not finish within the [request wait limit](#waiting-for-runs-and-builds). The run keeps going. |
| 409 | `task-name-exists` / `version-exists` | Conflicts with an existing name or version. |
| 409 | `idempotency-key-in-progress` | A run start with the same `Idempotency-Key` is still in progress. Retry shortly. |
| 422 | `idempotency-key-reused` | The `Idempotency-Key` was already used for a different request (another Actor, task, or build target). |
| 413 | `request-too-large` | The body is over the size limit. |
| 415 | `unsupported-content-encoding` | Unsupported `Content-Encoding`. |
| 429 | `rate-limit-exceeded` | Too many requests. |
| 429 | `concurrency-limit-exceeded` / `run-capacity-check-busy` | You hit your concurrent run limit, or another run request from your account is being evaluated right now. |
| 503 | `rate-limit-unavailable` | The rate limiter is temporarily unavailable. Retry after one second. |
| 5xx | `internal-error` | Unexpected server error. |

Retry `429` and `503` with exponential backoff: start near 500 ms, double each time, and add jitter. The [API clients](#api-clients) do this for you.

## Response headers

Rate-limited routes send these headers on both successful and rejected responses:

| Header | Meaning |
| --- | --- |
| `X-RateLimit-Limit` | Maximum requests in the current window. |
| `X-RateLimit-Remaining` | Requests left in the current window. |
| `X-RateLimit-Reset` | Unix time in seconds when the window resets. |
| `Retry-After` | Seconds to wait. Sent only with `429` and `503 rate-limit-unavailable`. |

Creation endpoints usually return `Location` with the new resource URL. Dataset-item responses, including synchronous dataset-item runs, also send pagination in `X-Apify-Pagination-Total`, `X-Apify-Pagination-Offset`, `X-Apify-Pagination-Count`, `X-Apify-Pagination-Limit`, and `X-Apify-Pagination-Desc`. The header names are historical.

The API allows cross-origin requests (CORS) from any origin, so you can call it from a browser. Keep in mind that a token used in browser code is visible to anyone who loads the page.

## Rate limits

Limits use a sliding one-minute window.

| Scope | Default limit |
| --- | --- |
| General `/v2` API with a valid token | **1000 requests / minute per account** |
| General `/v2` API without a valid token | **1000 requests / minute per IP address** |
| Datasets, key-value stores, and request queues | **5000 requests / minute per account**, plus a ceiling of 5000 per IP address |
| `/v2/api-keys` | **10 requests / minute per IP address** |
| Hosted MCP server (`/mcp`) | **1000 requests / minute per IP address** |
| [LLM gateway](https://scrape.ly/docs/llm-gateway.md#limits) (`/v2/llm/api/v1`), with a run token | **600 requests / minute** and **20 running at once, per account**. Not counted toward the general `/v2` limit |
| `GET /health` | **60 requests / minute per IP address** |

Calls an Actor makes to update or restart its own run (`PUT /v2/actor-runs/{runId}`, reboot, metamorph) count against the storage limit rather than the general one.

When you go over a limit, the API responds with `429 rate-limit-exceeded` and the headers above. Your account's limits can be raised. Contact support if you need more.

## Request size limits

| Body type | Limit |
| --- | --- |
| `application/json` | **10 MB** |
| `application/octet-stream` | **100 MB** |
| `text/*` | **100 MB** |

The [LLM gateway](https://scrape.ly/docs/llm-gateway.md) accepts JSON bodies up to **50 MB**, for inline images and files.

A body over the limit is rejected with `413 request-too-large` before it reaches a route handler. Split large dataset writes into several `POST /v2/datasets/{datasetId}/items` calls instead of one huge array, and send binary key-value records as `application/octet-stream`.

Request bodies may be compressed with the `identity`, `gzip`, `deflate`, or `br` content encodings. A compressed body may be at most **10 MB** before decompression, and the decompressed body must still fit the limit above. Any other `Content-Encoding` returns `415 unsupported-content-encoding`.

## Referring to resources

- **Resource ID**: an opaque ID. Actors and runs use 17-character IDs; some resources use UUIDs.
- **Named ID**: `username~resourcename`, for example `janedoe~my-actor`. You need the right permissions on the resource. Task endpoints also accept `~taskname` for one of your own tasks.

Most Actor paths use `/v2/actors`. These paths also exist under `/v2/acts`: `GET /v2/acts/{actorId}`, `GET /v2/acts/{actorId}/picture`, `GET /v2/acts/{actorId}/webhooks`, and the public example-task endpoints `GET /v2/acts/{actorId}/examples` and `GET /v2/acts/{actorId}/examples/{slug}`.

## Content-Type

JSON request bodies must send `Content-Type: application/json`:

```bash
curl -X POST https://api.scrape.ly/v2/actors/janedoe~my-actor/runs \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'
```

The run-start body is the **input object itself**, not `{ "input": ... }` or `{ "runInput": ... }`.

## Run and build statuses

Runs: `READY`, `RUNNING`, `SUCCEEDED`, `FAILED`, `ABORTING`, `ABORTED`, `TIMING-OUT`, `TIMED-OUT`.

Builds: `READY`, `RUNNING`, `SUCCEEDED`, `FAILED`, `TIMED-OUT`, `ABORTED`.

A resurrected run goes back to `RUNNING`. `RESURRECTED` only appears as the `ACTOR.RUN.RESURRECTED` webhook event.

Run statuses use a **hyphen**: `TIMED-OUT`. Webhook event names use an **underscore**: `ACTOR.RUN.TIMED_OUT` (see [Webhooks](https://scrape.ly/docs/api-webhooks.md)).

## Waiting for runs and builds

`waitForFinish` and the synchronous run endpoints hold the request open while the run or build works. One request is held for at most **45 seconds**. This is the server's `API_MAX_REQUEST_WAIT_SECS` setting, which keeps every response ahead of the 50-second limit of the proxy in front of the API.

- `waitForFinish` accepts up to 60, but the wait stops at 45 seconds. You then get the run or build in its current state, so check `status` before you read results.
- A synchronous run (`run-sync`, `run-sync-get-dataset-items`) that is still going after 45 seconds returns `408`. The run keeps going, and the error message includes its run ID.
- To wait longer, repeat [GET /v2/actor-runs/{runId}?waitForFinish=60](https://scrape.ly/docs/api-runs.md#get-run) until `status` is terminal. Each request returns as soon as the run finishes, or after 45 seconds. `call()` and `waitForFinish()` in the official clients already work this way.
- For long jobs, prefer [webhooks](https://scrape.ly/docs/api-webhooks.md) to waiting.

Don't retry a run start just because it timed out on the network. The run may already exist. Send an [`Idempotency-Key`](https://scrape.ly/docs/api-actors.md#idempotent-run-starts) so a retry returns the same run.

## Typical workflow

1. Start a run with [POST /v2/actors/{actorId}/runs](https://scrape.ly/docs/api-actors.md#start-a-run).
2. Poll [GET /v2/actor-runs/{runId}](https://scrape.ly/docs/api-runs.md#get-run) with `waitForFinish` until `status` is terminal, or use a [synchronous run](https://scrape.ly/docs/api-actors.md#run-actor-synchronously) for jobs that finish within 45 seconds.
3. Read tabular results from `defaultDatasetId` with [Get dataset items](https://scrape.ly/docs/api-datasets.md).
4. Read named records such as `OUTPUT` from `defaultKeyValueStoreId` with [Get record](https://scrape.ly/docs/api-key-value-stores.md).

For a task-oriented walkthrough, see [Run Actors](https://scrape.ly/docs/run-actors.md) and [Get results](https://scrape.ly/docs/results.md).

## API clients

You don't have to call the REST API by hand. These official tools wrap it:

- **JavaScript / TypeScript**: [`scrapely-client`](https://scrape.ly/docs/client-js.md), a typed client for the whole API with retries, pagination helpers, and run waiting. It works in Node.js and the browser.
- **Python**: [`scrapely-client-python`](https://scrape.ly/docs/client-python.md), the same client for Python, with sync and async interfaces.
- **MCP**: the [hosted MCP server](https://scrape.ly/docs/mcp.md) lets AI agents search the Store, run Actors, and read results.
- **CLI**: [`scrapely`](https://scrape.ly/docs/cli.md) handles login, deploys, runs, and storage from the terminal. `scrapely api` sends a raw authenticated request to any endpoint.

The Actor SDKs ([Node.js](https://scrape.ly/docs/sdk-node.md), [Python](https://scrape.ly/docs/sdk-python.md)) are for writing code that runs inside an Actor. To call the API from your own app, use an API client.

## Reference

- [How to run an Actor](https://scrape.ly/docs/api.md)
- [Actors](https://scrape.ly/docs/api-actors.md)
- [Actor runs](https://scrape.ly/docs/api-runs.md)
- [Actor tasks](https://scrape.ly/docs/api-tasks.md)
- [Datasets](https://scrape.ly/docs/api-datasets.md)
- [Key-value stores](https://scrape.ly/docs/api-key-value-stores.md)
- [Request queues](https://scrape.ly/docs/api-request-queues.md)
- [Webhooks](https://scrape.ly/docs/api-webhooks.md)
- [Automations](https://scrape.ly/docs/api-automations.md)
- [Schedules](https://scrape.ly/docs/api-schedules.md)
- [Users](https://scrape.ly/docs/api-users.md)
- [Store, reviews & issues](https://scrape.ly/docs/api-store.md)
- [Earnings](https://scrape.ly/docs/api-earnings.md)
- [Messages & notifications](https://scrape.ly/docs/api-messaging.md)
- [Search, alerts & utilities](https://scrape.ly/docs/api-misc.md)
- [Standby mode](https://scrape.ly/docs/api-standby.md)

---

- Previous: [Publish & monetize](https://scrape.ly/docs/publishing.md)
- Next: [API guide](https://scrape.ly/docs/api.md)
- All docs: [Docs index](https://scrape.ly/docs.md), [llms.txt](https://scrape.ly/llms.txt)