---
title: "MCP server · Scrape.ly Docs"
description: "Connect Claude, Cursor, VS Code, and other MCP clients to Actors, runs, and storage."
url: https://scrape.ly/docs/mcp
markdown: https://scrape.ly/docs/mcp.md
---

# MCP server

> Connect Claude, Cursor, VS Code, and other MCP clients to Actors, runs, and storage.

The Scrape.ly MCP server lets AI agents and MCP clients such as Claude, Cursor, and VS Code search the Store, run Actors, check runs, and read results from your account. Use the hosted endpoint for most clients. A local stdio server is available as a Python package for clients that cannot connect to a remote server.

The source code is on GitHub: [scrapely-app/scrapely-mcp](https://github.com/scrapely-app/scrapely-mcp).

## Hosted endpoint

Connect to `https://api.scrape.ly/mcp`. The endpoint uses the MCP streamable HTTP transport. Every request needs a Scrape.ly API token or an OAuth access token. Without one the server answers `401` with a `WWW-Authenticate` header that points MCP clients to the OAuth metadata.

### Authentication

Use one of:

- **OAuth.** Clients that support MCP authorization discover the flow on their own, register themselves, and open a browser window where you sign in to the dashboard and approve access. You do not need to copy a token.
- **Bearer token.** Send `Authorization: Bearer <token>` with a token from **Settings → API & Integrations** in the dashboard.
- **Query parameter.** `https://api.scrape.ly/mcp?token=<token>` for clients that cannot set headers. URLs end up in logs and history, so prefer a header.

Agents without a Scrape.ly account can pay per call with USDC instead: see [Pay with x402](https://scrape.ly/docs/x402.md).

The MCP server acts with your token's permissions: an agent can do anything the token can, including starting paid runs.

To limit what an agent can do, give it a scoped API key from **Settings → API keys**, for example the **Read only** preset, or `actors:run` + `runs:read` with a daily charge limit. See [API keys & permissions](https://scrape.ly/docs/api-keys.md).

### OAuth details

For client authors: the API implements OAuth 2.1 with PKCE and dynamic client registration.

```
GET  /.well-known/oauth-protected-resource
GET  /.well-known/oauth-authorization-server
POST /v2/oauth/register
GET  /v2/oauth/authorize
POST /v2/oauth/token
POST /v2/oauth/revoke
```

- Protected resource metadata (RFC 9728) and authorization server metadata (RFC 8414) are served from `https://api.scrape.ly`. The protected resource metadata is also available at the path form `/.well-known/oauth-protected-resource/mcp`.
- Registration (RFC 7591) needs `redirect_uris`. Each must be `https`, or `http` on `localhost` or `127.0.0.1`. Clients are public: `token_endpoint_auth_method` is `none`.
- `authorize` requires PKCE with `code_challenge_method=S256`.
- `token` supports `grant_type=authorization_code` and `grant_type=refresh_token`. Authorization codes are valid for 10 minutes.
- Access tokens start with `oat_` and expire after 24 hours. The client renews them with its refresh token, so you stay signed in.
- Refresh tokens start with `ort_` and are valid for 365 days from their last use: every refresh returns a new refresh token with a new 365-day window (`refresh_token_expires_in` in the token response). A client used at least once a year never has to sign in again.
- A refresh token works only for the client it was issued to. Each one can be used once; a repeat within 30 seconds returns the same new tokens, and a later reuse revokes every token from that sign-in.
- Revoking the app, changing your password, a suspension, deleting the account or resetting your API key revokes all refresh tokens. The client then asks you to sign in again.
- Client registrations unused for 366 days are deleted. The token endpoint then returns `invalid_client` (HTTP 401) and the client registers again.
- `revoke` accepts an access token or a refresh token in the `token` field. A refresh token revokes every token from that sign-in.

## Connect a client

Replace `YOUR_TOKEN` with your API token, or drop the header to sign in with OAuth in clients that support it.

### Claude Code

```bash
claude mcp add --transport http scrapely https://api.scrape.ly/mcp \
  --header "Authorization: Bearer YOUR_TOKEN"
```

Without `--header`, run `/mcp` inside Claude Code and choose the server to sign in with OAuth.

### Claude Desktop

Add the hosted endpoint as a custom connector under **Customize → Connectors** (on Team and Enterprise, an owner adds it under **Organization settings → Connectors**) using the URL `https://api.scrape.ly/mcp`. Claude signs you in with OAuth.

To run the [local server](https://scrape.ly/docs/mcp.md#local-server) instead, add it to `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "scrapely": {
      "command": "uvx",
      "args": ["--from", "scrapely-mcp@latest", "scrapely-mcp-server"],
      "env": {
        "SCRAPELY_TOKEN": "YOUR_TOKEN",
        "SCRAPELY_API_BASE_URL": "https://api.scrape.ly"
      }
    }
  }
}
```

### Cursor

Add the server to `~/.cursor/mcp.json`, or to `.cursor/mcp.json` in a project:

```json
{
  "mcpServers": {
    "scrapely": {
      "url": "https://api.scrape.ly/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_TOKEN"
      }
    }
  }
}
```

Without `headers`, Cursor offers to sign in with OAuth.

### VS Code

Add the server to `.vscode/mcp.json` in your workspace. This version prompts for the token once and stores it securely:

```json
{
  "inputs": [
    {
      "type": "promptString",
      "id": "scrapely-token",
      "description": "scrape.ly API token",
      "password": true
    }
  ],
  "servers": {
    "scrapely": {
      "type": "http",
      "url": "https://api.scrape.ly/mcp",
      "headers": {
        "Authorization": "Bearer ${input:scrapely-token}"
      }
    }
  }
}
```

### Other clients

Any client that supports streamable HTTP can use `https://api.scrape.ly/mcp` with a Bearer header or OAuth. Clients that only support stdio can use the local server.

## Tools

The server enables the `actors`, `docs`, `runs`, `storage`, `tasks`, and `automations` tool categories by default.

### actors

| Tool | What it does |
| --- | --- |
| `search-actors` | Search the [Store](https://scrape.ly/docs/store.md) by keywords. Returns up to 10 Actors (default 5). Only lists Actors whose owners allow AI agents (see [Store for AI agents](https://scrape.ly/docs/api-store.md#store-for-ai-agents)). |
| `fetch-actor-details` | Details for one Actor by ID or `username/actor-name`: description, input schema, README, pricing, stats, rating, and output schema. The input schema and README come from the build you run: on someone else's Actor, the [User Build](https://scrape.ly/docs/publishing.md#user-build). The `output` argument selects which parts to return. |
| `call-actor` | Run any Actor with a JSON `input`. Waits up to `waitSecs` (0–45, default 30), then returns the run status and storage IDs. Optional `callOptions`: `memory` (MB), `timeout` (seconds), `build` (your own Actors only), `maxItems`, `maxTotalChargeUsd`. |

### runs

| Tool | What it does |
| --- | --- |
| `get-actor-run` | Status, storages, and stats of a run, with a suggested next step. Waits up to `waitSecs` (0–45, default 30) for the run to finish. |
| `get-actor-output` | The results of a run: a page of its dataset items plus its key-value store `OUTPUT` record. See [Reading results](#reading-results). Also in the `storage` category. |
| `get-actor-run-list` | Your runs, filterable by `status`. Up to 10 per call. |
| `get-actor-log` | The last `lines` lines of a run's log (default 10, `0` for the whole log). |
| `abort-actor-run` | Abort a running run, optionally `gracefully`. |

### storage

| Tool | What it does |
| --- | --- |
| `get-actor-output` | Results of a run or dataset, paged and kept compact. See [Reading results](#reading-results). |
| `get-dataset-items` | Rows from a dataset. Supports `offset`, `limit` (default 20, up to 1000), `fields`, `omit`, `flatten`, `desc`, and `clean`. |
| `get-dataset` | Dataset metadata: field list, item count, and size. |
| `get-dataset-schema` | JSON schema inferred from a sample of items. |
| `get-dataset-list` | Your datasets. Up to 20 per call; `unnamed: true` includes run datasets. |
| `get-key-value-store` | Key-value store metadata. |
| `get-key-value-store-keys` | Keys in a store. Up to 10 per call. |
| `get-key-value-store-record` | One record by key. Records larger than 256 KiB are returned as a link instead of inline content. |
| `get-key-value-store-list` | Your key-value stores. Up to 10 per call. |

Datasets and stores accept an ID or `username~name`.

### tasks

| Tool | What it does |
| --- | --- |
| `create-actor-task` | Save an Actor's input and run options as a task. The `build` option only applies to your own Actors. |
| `get-actor-task` | Read a task. Secret input fields come back encrypted. |
| `update-actor-task` | Change a task's input, run options, or public page settings. The new input replaces the old one. The `build` option only applies to your own Actors. |
| `publish-actor-task` / `unpublish-actor-task` | Publish a task as a public page for its Actor, or take it down. |

### automations

| Tool | What it does |
| --- | --- |
| `list-saved-inputs` | Your [saved inputs](https://scrape.ly/docs/run-actors.md#saved-inputs). With `actorId`, one Actor's saved inputs, the default first, plus its example inputs; without it, all saved inputs in your account. |
| `run-saved-input` | Start a run of a saved input by `savedInputId`. `input` changes only the fields it names; the other fields come from the saved input. Returns right after the start, or waits up to `waitSecs` (at most 45). |
| `list-automations` | Your [automations](https://scrape.ly/docs/automations.md), with their trigger, steps, AI step, and deliveries. Filter with `actorId` or `savedInputId`. |
| `run-automation` | Run one of your automations now, whatever its trigger. Its runs are billed like any run, and an AI step like the AI chat. Returns the automation run with the started Actor runs. |

Agents can't create or change automations over MCP; use the dashboard, the AI chat, or the [Automations API](https://scrape.ly/docs/api-automations.md). On a scoped key, `list-automations` needs `automations:read`, and `run-automation` needs `automations:write` with `actors:run` and the run permission `actor-default` (see [API keys & permissions](https://scrape.ly/docs/api-keys.md#scopes)).

### docs

| Tool | What it does |
| --- | --- |
| `search-scrapely-docs` | Full-text search over Scrape.ly or Crawlee docs (`docSource`: `scrapely`, `crawlee-js`, or `crawlee-py`). |
| `fetch-scrapely-docs` | Fetch a docs page by URL from `docs.scrape.ly` or `crawlee.dev`. |

### Other tools

- `report-problem` sends a short problem report about a tool or Actor to the Scrape.ly team. It is in the `dev` category, which is off by default, and is never offered to Claude clients.
- Clients that support MCP Apps also get interactive variants of `search-actors`, `fetch-actor-details`, `call-actor`, and `get-actor-run`.

## Typical flow

1. `search-actors` finds candidate Actors for the task.
2. `fetch-actor-details` returns the input schema.
3. `call-actor` starts the run. Fast Actors finish within `waitSecs`; for long ones, the agent passes `waitSecs: 0` and polls `get-actor-run`.
4. `get-actor-output` reads the results: the run's dataset items, page by page, and its `OUTPUT` record.

Results are not inlined into the `call-actor` response. The agent reads them with `get-actor-output` (or the storage tools), so it can page through large datasets and pick only the fields it needs.

### Reading results

`get-actor-output` takes a `runId` (it uses the run's default dataset and key-value store) or a `datasetId`. It reads with your token, so you only see your own runs and storages.

| Argument | Default | Description |
| --- | --- | --- |
| `runId` / `datasetId` | — | One of them is required. |
| `fields` | all | Comma-separated fields to keep, with dot notation for nested values (`address.city`). |
| `offset` | `0` | Items to skip. |
| `limit` | `100` | Items per page, up to 1000. |
| `includeOutputRecord` | `true` with a `runId` or `keyValueStoreId` | Also return the key-value store `OUTPUT` record. |
| `keyValueStoreId` | the run's store | Read `OUTPUT` from this store instead. |

To keep replies small for the model, text values over 1,000 characters are cut, a page holds about 50 KB (so it can have fewer items than `limit`), and the `OUTPUT` record is cut at about 20 KB. Binary records are described, not inlined. The reply says `totalItemCount`, `hasMore`, `nextOffset`, and whether anything was `truncated`; pass `nextOffset` as `offset` for the next page.

## Local server

The Python package `scrapely-mcp` runs the same server over stdio on your machine. It requires Python 3.11 or newer and version 0.2.0 or later of the package. The command is `scrapely-mcp-server`:

The `automations` tools come with version 0.3.0 of the package.

```bash
uvx --from scrapely-mcp@latest scrapely-mcp-server
```

or

```bash
pip install -U scrapely-mcp
scrapely-mcp-server
```

> [!NOTE]
> Since 0.2.0 the server connects to `https://api.scrape.ly` by default. Releases before 0.2.0 defaulted to a non-production API URL; if you still run one, upgrade or set `SCRAPELY_API_BASE_URL=https://api.scrape.ly`.

### Environment variables

| Variable | Description |
| --- | --- |
| `SCRAPELY_TOKEN` | API token. If unset, the server reads `token` from `~/.scrapely/credentials.json` or `~/.scrapely/auth.json`, which the [CLI](https://scrape.ly/docs/cli.md) writes when you log in. |
| `SCRAPELY_API_BASE_URL` | API origin. Defaults to `https://api.scrape.ly`. |

A token is required unless every selected tool works without one (`search-actors`, `fetch-actor-details`, and the docs tools). If a token is required and none is found, the server exits with an error.

### Options

| Flag | Description |
| --- | --- |
| `--tools` | Comma-separated categories (`actors`, `docs`, `runs`, `storage`, `tasks`, `automations`, `dev`) or individual tool names. Replaces the default set. |
| `--actors` | Comma-separated Actors (`username/actor-name`) to expose as their own tools. |
| `--ui` | `auto` (default) enables MCP Apps widgets when the client supports them; `apps` forces them; `default` turns them off. Widgets are disabled in the current release, so every client gets the plain tools. |
| `--telemetry-enabled` | Send anonymous usage telemetry. Off by default. |

```bash
# Only docs search, no token needed
scrapely-mcp-server --tools=docs

# Runs and storage only
scrapely-mcp-server --tools=runs,storage
```

Whenever `call-actor`, `get-actor-run`, or an Actor tool from `--actors` is enabled, the server also adds `get-actor-run`, `get-actor-output`, `get-dataset-items`, `get-key-value-store-record`, and `abort-actor-run` so the agent can follow up on runs.

## Costs and limits

- Actor runs started through MCP are normal runs on your account. They use your plan's compute and storage, and paid Actors charge as usual. See [Account](https://scrape.ly/docs/account.md).
- Bound spending with the Actor's own input fields (such as a result limit), or with `callOptions.maxItems` for Actors that charge per result item and `callOptions.maxTotalChargeUsd` for any pay-per-event Actor.
- `callOptions.memory` accepts 128 to 32,768 MB. Your plan also caps total memory across concurrent runs; a run that would exceed it is rejected.
- `call-actor` and `get-actor-run` wait at most 45 seconds per call. Runs keep going after the tool returns.
- The hosted endpoint allows 1,000 requests per minute per IP address.
- On the hosted endpoint, `?tools=` and `?actors=` pick the tools the same way as `--tools` and `--actors`, for example `https://api.scrape.ly/mcp?actors=janedoe/my-actor`. One selection can name at most 50 Actors; more returns `400` with the error "Too many Actors selected".
- Each token can have 50 open sessions on the hosted endpoint, and each IP address 100. Past that, your oldest idle session is closed, or the request gets `429`.
- A tool result can be at most 4 MiB. Clients that run tools as protocol tasks get finished task results for up to 15 minutes after the task ends, or less when the task's `ttl` is shorter. A result is released a minute after it's fetched, and closing the session drops its tasks.

---

- Previous: [Python client](https://scrape.ly/docs/client-python.md)
- Next: [Pay with x402](https://scrape.ly/docs/x402.md)
- All docs: [Docs index](https://scrape.ly/docs.md), [llms.txt](https://scrape.ly/llms.txt)