Config, CI/CD & troubleshooting

This page collects everything that configures the CLI: the help and upgrade commands, the files it reads and writes, and the environment variables it honours. It also shows how to run the CLI in CI pipelines and how to fix common errors.

Help

scrapely help
scrapely help push
scrapely help "actors ls"
scrapely push --help

scrapely help prints an overview of the command groups. With a command, including a nested one in quotes, it prints that command's options. --help after any command does the same.

Upgrade the CLI

scrapely upgrade
scrapely upgrade --version 0.2.0

upgrade compares the installed version with the latest one on npm (npm view scrapely-cli version) and runs npm install -g scrapely-cli@latest if yours is older.

If npm needs elevated permissions for global installs on your machine, upgrade fails with a permissions error. Run npm install -g scrapely-cli@latest yourself in that case.

Files

In your home folder

The CLI keeps its own state in ~/.scrapely/. Both files are written with mode 0600.

In your project

The Actor config is the first of these files that exists in the project folder:

1. .actor/actor.json
2. .actor/scrapely.json
3. scrapely.json
4. actor.json

If the config is in .actor/ and the folder also has a root scrapely.json, the root file's fields override those in .actor/. Paths in the config, such as input or dockerfile, are relative to the file that contains them. package.json provides name, version, and actorId when the config doesn't. The fields are described in Projects & running.

Other project files the CLI reads:

Environment variables

For CLI commands

A saved token wins over SCRAPELY_TOKEN, but the API URL variables win over the saved API URL. Without any API URL, the CLI uses its built-in default, which is not the production API. Set SCRAPELY_API_URL=https://api.scrape.ly and SCRAPELY_WEB_URL=https://my.scrape.ly. See Authentication.

For runtime commands

The scrapely actor commands also read ACTOR_TOKEN, APIFY_TOKEN, and APIFY_API_BASE_URL, and the run and storage IDs the platform sets in each container. The other commands ignore these. See Actor runtime.

Set for your code by scrapely run

run sets ACTOR_INPUT (the input as base64-encoded JSON), ACTOR_INPUT_KEY=INPUT, and the variables from the config's environmentVariables.

CI/CD

In a pipeline, authenticate with an API key in SCRAPELY_TOKEN. Don't run scrapely login: without --token it waits for input. Browser-login tokens expire, so don't copy one from your machine into CI.

GitHub Actions

name: Deploy Actor
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '22'

      - name: Install CLI
        run: npm install -g scrapely-cli

      - name: Check schemas
        run: scrapely validate-schema

      - name: Deploy
        env:
          SCRAPELY_API_URL: https://api.scrape.ly
          SCRAPELY_TOKEN: ${{ secrets.SCRAPELY_TOKEN }}
        run: scrapely push --force --wait-for-finish 900

GitLab CI

deploy:
  stage: deploy
  image: node:22
  variables:
    SCRAPELY_API_URL: https://api.scrape.ly
  script:
    - npm install -g scrapely-cli
    - scrapely validate-schema
    - scrapely push --force --wait-for-finish 900
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

Define SCRAPELY_TOKEN as a masked CI/CD variable in the project settings.

The first push writes the new actorId into the config file. Commit that change, or every pipeline run on a fresh checkout creates another Actor.

In CI, push can't ask before overwriting an Actor that changed on the platform after your files did, so it fails instead. Pass --force when the repository is the source of truth, as above.

Rebuild on Git push without a pipeline

If the Actor's source is a GitHub repository, the platform can rebuild it on every push to the linked branch without running the CLI. With the GitHub integration connected, linking the repository registers the webhook for you, and scrapely create --source github --auto-build on does it for new projects. To add or repair the webhook on an existing Actor, run scrapely api POST /v2/actors/<actorId>/ci/webhook -d '{}'. For other Git hosts, get the build URL with scrapely api GET /v2/actors/<actorId>/ci/webhook-url and add it as a webhook there. See Integrations.

Scripting tips

• Keep the token in your CI secret store. Never commit it.
• Pass -y to delete commands so nothing waits for confirmation.
• Use --json and jq instead of parsing the formatted output, which can change.
• Let push wait for the build, so a failed build fails the job. With -w <seconds>, a build still running at the deadline exits with code 0.
• Check exit codes: call, runs wait, and builds wait exit with code 1 unless the run or build SUCCEEDED.

Start a run, wait for it, and export the results:

RUN_ID=$(scrapely actors start janedoe~my-actor -i '{"query":"coffee"}' --json | jq -r '.data.id')
scrapely runs wait "$RUN_ID" --timeout 900
DATASET_ID=$(scrapely runs info "$RUN_ID" --json | jq -r '.data.defaultDatasetId')
scrapely datasets get-items "$DATASET_ID" --format csv > results.csv

Troubleshooting

Getting more detail

When a command fails, the CLI prints the API's error message under Details:. To see the raw response, repeat the request with scrapely api:

scrapely api GET /v2/users/me
scrapely api GET /v2/actors/janedoe~my-actor

What's next

• Authentication: tokens and precedence.
• Secrets, webhooks & API: the api command in full.
• API overview: status codes, errors, and rate limits.