---
title: "LLM gateway · Scrape.ly Docs"
description: "Call OpenRouter models from a run with its run token: endpoints, pricing, holds, limits, and errors."
url: https://scrape.ly/docs/llm-gateway
markdown: https://scrape.ly/docs/llm-gateway.md
---

# LLM gateway

> Call OpenRouter models from a run with its run token: endpoints, pricing, holds, limits, and errors.

Actors can call AI models without bringing their own API key. Scrape.ly runs an OpenRouter-compatible gateway with access to every model OpenRouter lists. A run calls it with its own run token as the API key, and each request is billed to the account that started the run as run usage.

```js [JavaScript]
const client = new OpenAI({ baseURL: process.env.SCRAPELY_LLM_BASE_URL, apiKey: process.env.SCRAPELY_TOKEN });
```

```python [Python]
client = OpenAI(base_url=os.environ["SCRAPELY_LLM_BASE_URL"], api_key=os.environ["SCRAPELY_TOKEN"])
```

> [!NOTE]
> The gateway works only from inside a run. Your account API key is refused, so the gateway can't be used as a general-purpose LLM API.

## Connect from an Actor

Every run gets the gateway's base URL in `SCRAPELY_LLM_BASE_URL` (it ends in `/v2/llm/api/v1`). Use the run token in `SCRAPELY_TOKEN` (also `APIFY_TOKEN`) as the API key. The gateway accepts it as `Authorization: Bearer <token>` or as `x-api-key: <token>`, so OpenAI, OpenRouter, and Anthropic SDKs all work.

```js [JavaScript]
import OpenAI from 'openai';
import { Actor } from 'scrapely';

await Actor.main(async () => {
    const client = new OpenAI({
        baseURL: process.env.SCRAPELY_LLM_BASE_URL,
        apiKey: process.env.SCRAPELY_TOKEN,
    });

    const completion = await client.chat.completions.create({
        model: 'openai/gpt-4o-mini',
        messages: [{ role: 'user', content: 'Summarize this page in one sentence: ...' }],
        max_tokens: 500,
    });

    await Actor.pushData({ summary: completion.choices[0].message.content });
});
```

```python [Python]
import asyncio
import os

from openai import AsyncOpenAI
from scrapely import Actor


async def main() -> None:
    async with Actor:
        client = AsyncOpenAI(
            base_url=os.environ["SCRAPELY_LLM_BASE_URL"],
            api_key=os.environ["SCRAPELY_TOKEN"],
        )
        completion = await client.chat.completions.create(
            model="openai/gpt-4o-mini",
            messages=[{"role": "user", "content": "Summarize this page in one sentence: ..."}],
            max_tokens=500,
        )
        await Actor.push_data({"summary": completion.choices[0].message.content})


if __name__ == "__main__":
    asyncio.run(main())
```

Model IDs are OpenRouter's, in the form `author/model`, such as `openai/gpt-4o-mini`. List them with `GET /models`.

The Anthropic SDK adds `/v1/messages` to its base URL itself, so give it the URL without the trailing `/v1`:

```python
import os
from anthropic import Anthropic

client = Anthropic(
    base_url=os.environ["SCRAPELY_LLM_BASE_URL"].removesuffix("/v1"),
    api_key=os.environ["SCRAPELY_TOKEN"],
)
```

> [!WARNING]
> Don't point `OPENAI_BASE_URL` at the gateway in an Actor that also uses a user's own OpenAI key. The SDK would send that key to the gateway, which only accepts run tokens. That's why Scrape.ly sets `SCRAPELY_LLM_BASE_URL` and not `OPENAI_BASE_URL`.

### Actors built for Apify

Apify Actors call `https://openrouter.apify.actor/api/v1` with their run token. Scrape.ly serves the same paths, so every build replaces that address with the Scrape.ly gateway. This happens whatever the **Transform actors** setting says, so a run token is never sent to Apify:

- Only the build's temporary copy of the source changes. Your stored source is never modified.
- Files scanned: `.py`, `.js`, `.mjs`, `.cjs`, `.ts`, `.mts`, `.cts`, `.tsx`, `.jsx`, `.json`, `.yaml`, `.yml`, `.toml`, `.env`, `.ini`, and `.cfg`, up to 5 MB each. `node_modules`, `.git`, virtual environments, and symlinks are skipped.
- Only the literal text `https://openrouter.apify.actor` is replaced. An address built at runtime, for example from pieces or from an input field, is left alone.
- When anything changed, the build log shows `[Scrapely Build] LLM URL rewrite: N file(s) (openrouter.apify.actor → Scrapely LLM gateway)`.

For an address built at runtime, use `SCRAPELY_LLM_BASE_URL` as the base URL.

## Endpoints

All paths are relative to `SCRAPELY_LLM_BASE_URL`, and they match OpenRouter's paths after `/api/v1`.

| Method and path | What it does | Billed |
| --- | --- | --- |
| `POST /chat/completions` | OpenAI-style chat completions, streaming or not | Yes |
| `POST /responses` | OpenAI Responses API | Yes |
| `POST /messages` | Anthropic Messages API | Yes |
| `POST /embeddings` | Embeddings | Yes |
| `GET /models`, `GET /models/count` | Model catalogue with prices | No |
| `GET /models/{author}/{slug}/endpoints` | Providers serving one model | No |
| `GET /embeddings/models` | Embedding models | No |
| `GET /providers` | Provider list | No |

Any other path returns `404 not-found`. Free catalogue calls still count toward the [per-minute limit](#limits).

Streaming responses are passed through as they arrive. Request bodies can be up to **50 MB**, so inline images and PDFs as base64 fit.

## What the gateway changes in a request

The gateway forwards your request almost as is. It changes only what it needs to bill safely:

| Field | Change |
| --- | --- |
| `max_tokens`, `max_completion_tokens`, `max_output_tokens` | Capped at **65,536** output tokens (free plan: **8,192**), or lower if the model's own maximum is lower. With no limit set, the cap is set for you, reduced if needed so the prompt plus the output still fits the model's context window (never below 1,024). The limit applied is returned in the `X-Scrapely-Max-Output-Tokens` response header |
| `thinking.budget_tokens`, `reasoning.max_tokens` | Lowered to one below the output limit when they reach or exceed it |
| `n` | At most **8** choices |
| `models` | Removed. Model fallbacks would let a cheap model front an expensive one, so send one `model` |
| `user` | Set to your Scrape.ly user ID |
| `usage`, `stream_options.include_usage` | `usage` is turned on for chat completions, plus `stream_options.include_usage` when streaming, so the cost comes back with the response |

Reasoning tokens count against the output limit, so leave room for thinking plus the answer.

For `openrouter/auto` and other routers whose price depends on the model they pick, a request with no output limit gets **8,192** tokens.

## Pricing

Each request costs exactly what OpenRouter charges for it, times a markup, rounded up only to the next nanodollar ($0.000000001):

| Plan | Price |
| --- | --- |
| Paid | OpenRouter's cost × **1.25** |
| Free | OpenRouter's cost × **1.2**, with a daily limit (see [Free plan](#free-plan)) |

Your plan's markup is shown in **Billing → Pricing** under **AI models**, and in `GET /v2/users/me` as `plan.planPricing.llmGatewayMarkup`.

OpenRouter's cost covers everything it bills for: input and output tokens, cached tokens, reasoning, web search, images, and PDF parsing. The price per token depends on the model; `GET /models` lists it. Failed requests that OpenRouter doesn't charge for aren't billed, and neither are free models (`:free`).

Where it shows up:

| Where | What you see |
| --- | --- |
| **Billing → Current period** | **AI model tokens** under **Actors**, with the token count and cost |
| **Runs → run → Cost breakdown** | **AI model tokens**, with the token count and cost. It's included in the run's total cost in the runs list |
| Run object in the API | `usage.LLM_TOKENS` (input plus output tokens) and `usageUsd.LLM_TOKENS` |
| `GET /v2/users/me/plan` | `usage.usageUsd.llm`, also included in `usageTotalUsd` |
| `GET /v2/users/me/usage/monthly` | `LLM_TOKENS` in `monthlyServiceUsage` and in each day's breakdown. Prices differ by model, so its `baseUnitPriceUsd` is the average price per token across the models used |

### How a request is paid for

1. **Hold.** Before the request goes to OpenRouter, the gateway holds its worst-case cost from your balance: the prompt, the full output limit for every choice, and any web search, per-image, or PDF-parsing charges, plus a 25% margin. For `openrouter/auto`, the hold uses the 95th-percentile price of all listed models for each price type (prompt, output, per request), not the most expensive model's. If the hold doesn't fit, the request is refused with `402` and nothing is sent.
2. **Charge.** After the response ends, the run is charged OpenRouter's reported cost × the markup.
3. **Release.** The hold is released once the charge is booked, so the money stays covered the whole time.

Holds count as spent while they last. Several large requests in parallel can therefore get a `402` even when the actual costs would fit your balance, and starting a new run sees less spendable balance while LLM requests are in flight. The run's own compute reservation counts as available for its LLM calls.

### Cancelled and cut-off streams

A request without `stream: true` is always read to the end and billed in full, even if your code stops waiting for it.

If your code stops reading a stream (closes the connection, times out, or the run ends), the gateway cancels the request upstream. Providers that support cancellation stop generating, and you pay only for what was generated. Some providers keep generating to the end, and OpenRouter bills the full output.

In both cases the gateway bills from OpenRouter's record of the request, not from what reached your code. That record can take a while to appear, up to the full request time for providers that keep generating. Meanwhile the hold stays in place, and the charge can show up on the run up to about 30 minutes later. If the record never appears, the gateway bills its own estimate.

### Spending cap and balance

- LLM charges count toward the run's `maxTotalChargeUsd`. A request whose hold would go past the run's remaining budget is refused with `402 charge-limit-exceeded` before anything is sent. If a charge still reaches the cap, it's cut to what fits and the run is stopped.
- If your balance can't pay a charge when it's booked, the run is aborted with `Run aborted: Insufficient credits to pay for LLM usage. Please add credits.` The charge stays recorded, and the platform retries it for about 10 minutes, so adding credits right away pays it.

## Free plan

Free-plan accounts can use the gateway with tighter limits:

- **$2.50 per day** of LLM charges (after the markup), counted per UTC day across all your runs. In-flight holds count toward it. Past it, requests get `402 llm-free-daily-limit` until the next day.
- Output capped at **8,192** tokens per request. The first time a run's request is cut down to this limit, or sent with no limit and given it, the run log shows `[Scrapely] LLM output capped at 8,192 tokens per request (free plan). Upgrade to raise it.` Every chat, responses, and messages response also carries the limit in the `X-Scrapely-Max-Output-Tokens` header, and an answer cut off by it ends with `finish_reason: "length"`.
- 1.2× markup instead of 1.25×.

`GET /v2/users/me/limits` returns the limit and today's settled spend as `llmFreeDailyCapUsd` (`{ "used", "limit" }` in USD). It's `null` on a paid plan.

A paid plan removes the daily limit. Paid accounts are limited only by their balance, each run's `maxTotalChargeUsd`, and the [rate limits](#limits).

## Limits

| Limit | Default | Response when exceeded |
| --- | --- | --- |
| Requests per minute, per account | **600** | `429 rate-limit-exceeded` with `Retry-After` |
| Requests running at the same time, per account | **20** | `429 rate-limit-exceeded`: `Too many concurrent LLM requests` |
| Requests with an invalid run token, per IP address | **1000 / minute** | `429 rate-limit-exceeded` |
| Request body | **50 MB** | `413 request-too-large` |
| Time without a byte from the model | **10 minutes** | Request ends |
| Total time per request, streaming included | **30 minutes** | Request ends |

The first two limits are per account and can be raised. They're shown in **Billing → Limits** as **LLM Gateway Rate Limit** and **LLM Gateway Concurrent Requests**, and in `GET /v2/users/me/limits` as `maxLlmRateLimit` and `maxLlmConcurrency`. Gateway requests don't count toward the general API limit.

A request stops counting toward the concurrency limit as soon as its response ends.

## Errors

Errors from the gateway use the OpenAI shape, so SDKs show the message:

```json
{ "error": { "type": "insufficient-balance", "message": "Insufficient balance for this LLM request. Please add credits to your account." } }
```

| Status | `type` | Cause |
| --- | --- | --- |
| 400 | `invalid-request` | Body isn't a JSON object, is invalid JSON, or has no `model` |
| 400 | `model-not-found` | The model isn't in `GET /models` or `GET /embeddings/models` |
| 401 | `token-not-provided` | No API key sent |
| 401 | `token-not-valid` | The run token is invalid, or the run has finished |
| 402 | `insufficient-balance` | The request's hold doesn't fit your balance |
| 402 | `charge-limit-exceeded` | The request could go past the run's `maxTotalChargeUsd` |
| 402 | `llm-free-daily-limit` | The free plan's daily LLM limit is reached |
| 403 | `llm-gateway-runs-only` | The key isn't a run token, for example your account API key |
| 404 | `not-found` | The path or method isn't served by the gateway |
| 413 | `request-too-large` | Body over 50 MB |
| 429 | `rate-limit-exceeded` | Per-minute or concurrency limit, or too many invalid tokens |
| 500 | `internal-error` | The gateway failed. Retry |
| 502 | `llm-upstream-error` | OpenRouter couldn't be reached or is unavailable, on any endpoint, including the model list. Retry |
| 503 | `llm-gateway-disabled` | The gateway is turned off on this platform |

Other errors from the model provider, such as an invalid parameter or a provider-side rate limit, are passed through with OpenRouter's status and body.

## What's next

- [Security](https://scrape.ly/docs/security.md#run-tokens): how run tokens work
- [Run Actors](https://scrape.ly/docs/run-actors.md#query-parameters): set `maxTotalChargeUsd` to cap a run's LLM spend
- [Account & billing](https://scrape.ly/docs/account.md): balance, usage, and limits
- [Publish and monetize](https://scrape.ly/docs/publishing.md#the-users-budget): LLM charges count toward a user's budget

---

- Previous: [Scrape.ly Browser](https://scrape.ly/docs/browser.md)
- Next: [Security](https://scrape.ly/docs/security.md)
- All docs: [Docs index](https://scrape.ly/docs.md), [llms.txt](https://scrape.ly/llms.txt)