CLI

Secrets, webhooks & API

Manage local secrets, configure webhooks, and call any API endpoint.

This page covers three things: local secrets that scrapely run passes to your code as environment variables, how to manage webhooks without a dedicated command, and scrapely api, which sends an authenticated request to any API endpoint.

Secrets

Secrets are values you don't want in your repository, such as API keys for the sites your Actor calls. The CLI stores them on your machine in ~/.scrapely/secrets.json, written with mode 0600. Your Actor config refers to them by name with an @ prefix:

.json

{
    "environmentVariables": {
        "LOG_LEVEL": "info",
        "TARGET_API_KEY": "@target-api-key"
    }
}

Add a secret

.bash

scrapely secrets add target-api-key
scrapely secrets add target-api-key "sk-live-..."

Without a value, add asks for it with masked input, which keeps it out of your shell history. A name may contain letters, digits, underscores, and hyphens. Adding a name that exists replaces its value.

List secrets

.bash

scrapely secrets ls
scrapely secrets ls --json

ls prints each name with its creation time. Values are never shown. --json prints { "secrets": [<names>] }.

Remove a secret

.bash

scrapely secrets rm target-api-key

rm deletes the secret immediately, without a prompt. It fails if the name doesn't exist.

How the CLI uses secrets

scrapely run sets every variable in environmentVariables for the local process and replaces each @name with the stored value. A missing secret stops the run, unless you pass --allow-missing-secrets, which skips the variable with a warning.
scrapely push checks that every @name exists in your local secrets and stops if one is missing, unless you pass --allow-missing-secrets. It doesn't upload the values.

Secrets live on one machine. A teammate or CI runner that pushes the same Actor needs its own copies, or --allow-missing-secrets.

Secrets on the platform

Local secrets never reach the platform. On the platform, a value that starts with @ in the config file's environmentVariables is passed to the run as the literal name without the @, so TARGET_API_KEY would contain target-api-key. Plain values in that block are passed as they are.

To give a run the real value, set the variable on the Actor version instead. In the dashboard, open the Actor, go to the Source tab, and add it under Environment Variables with Secret checked. A variable set on the version wins over the same name in the config file.

.bash

printf '{"name":"TARGET_API_KEY","value":"%s","isSecret":true}' "$TARGET_API_KEY" \
  | scrapely api POST /v2/actors/<actorId>/versions/0.0/env-vars -d -

Webhooks

The CLI has no webhooks command. Create and manage webhooks in the dashboard or with the Webhooks API, which you can call with scrapely api:

.bash

scrapely api GET /v2/webhooks
scrapely api GET /v2/webhooks/<webhookId>
scrapely api GET /v2/webhook-dispatches

For event types, payloads, and retries, see Integrations.

Call the API

scrapely api sends a request to any endpoint with your saved token. Use it for anything that has no dedicated command.

.bash

scrapely api [methodOrEndpoint] [endpoint]

The method can come first, or be set with -X. The default is GET. A path that doesn't start with /v2/ or /v1/ gets /v2 in front, so these are the same request:

.bash

scrapely api GET /v2/actors
scrapely api /v2/actors
scrapely api actors

Option	Description
`-X, --method <method>`	`GET`, `POST`, `PUT`, `PATCH`, or `DELETE`. Ignored when the method is the first argument.
`-d, --body <body>`	Request body. Must be JSON. `-` reads it from stdin.
`-p, --params <json>`	Query parameters as a JSON object
`-H, --header <header>`	Extra headers, as `Key:Value` (comma-separated for several) or a JSON object
`-l, --list-endpoints`	List the endpoints the CLI knows about
`-s, --search <query>`	Filter `--list-endpoints`
`--describe <path>`	Show the method and path parameters of one endpoint

JSON responses are pretty-printed. When the API returns an error, api prints the error body to stdout and exits with code 1.

.bash

# Your account
scrapely api GET /v2/users/me

# Actors you created, five at a time
scrapely api GET /v2/actors -p '{"my":true,"limit":5}'

# Start a run. The body is the input object itself.
scrapely api POST /v2/actors/janedoe~my-actor/runs -d '{"startUrls":[{"url":"https://crawlee.dev"}]}'

# Send a body from a file
scrapely api POST /v2/actors/janedoe~my-actor/runs -d - < input.json

# Find endpoints
scrapely api --list-endpoints --search dataset
scrapely api --describe /v2/actors/{actorId}/runs

The endpoint list is built into the CLI and may lag behind the API. The API reference is the full list. See API overview for the response envelope, errors, and rate limits.

What's next

Integrations: webhooks, schedules, and other ways to connect Actors.
Security: handle tokens and secrets safely.
API overview: what every request and response looks like.