CLI

Actor runtime

Commands for use inside a running Actor: input, output, storage, and charging.

The scrapely actor commands are for code running inside an Actor run. They read the run's input, write to its default dataset and key-value store, and charge pay-per-event events, using the environment variables the platform sets in the container. They let an Actor written as a shell script, or in a language without a Scrape.ly SDK, use the platform. generate-schema-types is the exception: it's a development tool you run on your machine.

In JavaScript and Python, use the SDK instead: Actor.getInput(), Actor.pushData(), Actor.setValue(), and so on. See the JavaScript SDK and the Python SDK.

scrapely actor is not scrapely actors. The plural group manages Actors in your account from your machine. See Actor management.

Command	Purpose
`actor get-input`	Print the run's input
`actor push-data [item]`	Append items to the default dataset
`actor get-value <key>`	Print a record from the default key-value store
`actor set-value <key> [value]`	Write or delete a record in the default key-value store
`actor get-public-url <key>`	Print the API URL of a record
`actor charge <eventName>`	Charge a pay-per-event event
`actor generate-schema-types [path]`	Generate TypeScript types from the input schema

Read the input

.bash

scrapely actor get-input
URL=$(scrapely actor get-input | jq -r '.startUrls[0].url')

get-input prints the input as JSON. On the platform, where ACTOR_DEFAULT_KEY_VALUE_STORE_ID is set, it reads the INPUT record from the run's default key-value store through the API. If that record doesn't exist or can't be read, it falls back to the ACTOR_INPUT environment variable, which can hold plain or base64-encoded JSON. Under scrapely run on your machine, it reads ACTOR_INPUT, which run sets. With neither available, it fails.

Write results

Push data

.bash

scrapely actor push-data '{"url":"https://example.com","title":"Example"}'
echo '[{"url":"https://a.example"},{"url":"https://b.example"}]' | scrapely actor push-data

push-data appends a JSON object, or an array of objects, to the run's default dataset. Without an argument it reads from stdin. Invalid JSON is rejected. The dataset is ACTOR_DEFAULT_DATASET_ID, or APIFY_DEFAULT_DATASET_ID.

To push many items, send one array rather than calling the command once per item. Each call is a separate API request.

Read a record

.bash

scrapely actor get-value STATE

get-value prints a record from the run's default key-value store. JSON records are pretty-printed. A missing key is an error.

Write a record

.bash

scrapely actor set-value OUTPUT '{"processed":42}'
scrapely actor set-value page.html "$(cat page.html)" -c text/html

Option	Description
`-c, --content-type <type>`	Content type of the record. Default `application/json`, which requires valid JSON.

An empty or missing value deletes the record:

.bash

scrapely actor set-value TEMPORARY

Get a record URL

.bash

scrapely actor get-public-url OUTPUT

get-public-url prints <API URL>/v2/key-value-stores/<storeId>/records/<key>?enablePublicUrl=true for the run's default store. It builds the URL locally, without calling the API.

Charge for events

.bash

scrapely actor charge result-scraped
scrapely actor charge result-scraped --count 25 --idempotency-key "batch-7"

charge charges an event of a pay-per-event Actor for the current run. The run is ACTOR_RUN_ID, or APIFY_ACTOR_RUN_ID.

Option	Description
`--count <number>`	Number of events. Default `1`.
`--idempotency-key <key>`	Key that makes a repeated charge count only once
`--test-pay-per-event`	Print what would be charged and send nothing. Needs no run ID.

Without --idempotency-key, every call gets a new key, so retrying a call after a network error charges the event twice. Pass a stable key whenever a charge can be retried. Pricing setup is covered in Publishing.

Generate TypeScript types

.bash

scrapely actor generate-schema-types
scrapely actor generate-schema-types .actor/input_schema.json -o src/types

generate-schema-types turns the Actor's input schema into TypeScript types. Run it on your machine in the project folder and commit the output.

Without a path, it uses the first of these files that exists: INPUT_SCHEMA.json, .actor/INPUT_SCHEMA.json, .actor/input_schema.json, input_schema.json, .actor/actor.json, .actor/scrapely.json, scrapely.json, actor.json. For a config file, it uses the schema in the config's input field, inline or by path.

It writes two files to src/__generated__/actor/:

input.ts, with an interface named after the schema's title in PascalCase (ActorInput if there is no title), also exported as Input.
index.ts, which re-exports both names.

Option	Description
`--all-optional`	Make every property optional
`-o, --output <dir>`	Output folder. It must be inside the current folder.
`--strict`	Don't add an index signature for objects that allow additional properties

Only the input schema is converted. Dataset and output schemas are ignored.

.ts

import { Actor } from 'scrapely';
import type { Input } from './__generated__/actor/index.js';

await Actor.init();
const input = (await Actor.getInput<Input>())!;

Environment

The platform sets these variables in every run. The runtime commands read them.

Variable	Used by
`ACTOR_RUN_ID`, `APIFY_ACTOR_RUN_ID`	`charge`. `push-data` when no dataset ID is set.
`ACTOR_DEFAULT_DATASET_ID`, `APIFY_DEFAULT_DATASET_ID`	`push-data`
`ACTOR_DEFAULT_KEY_VALUE_STORE_ID`, `APIFY_DEFAULT_KEY_VALUE_STORE_ID`	`get-input`, `get-value`, `set-value`, `get-public-url`
`ACTOR_INPUT`	`get-input` when the `INPUT` record can't be read

The token comes from the first of:

ACTOR_TOKEN
APIFY_TOKEN (set by the platform in every run)
SCRAPELY_TOKEN
The token saved by scrapely login

The API URL comes from the first of:

APIFY_API_BASE_URL (set by the platform in every run)
SCRAPELY_API_BASE_URL
SCRAPELY_API_URL
The API URL resolved as for other commands. See Authentication.

If any of the three API URL variables is set, the saved token is never used. A run with an API URL but no token variable fails instead of sending the token saved on the machine to that URL.

To try these commands on your machine, set the variables yourself, for example ACTOR_DEFAULT_DATASET_ID to a dataset you own, and keep using your saved login.

What's next

Storage management: work with any storage by ID, from outside a run.
JavaScript SDK and Python SDK: the same features from code.
Publishing: set up pay-per-event pricing.