Runs & builds

A build turns one version of an Actor's source into a Docker image. A run executes a build. The scrapely runs and scrapely builds commands list, inspect, wait for, and clean up both, and builds also starts builds and manages build tags.

Runs

List runs

scrapely runs ls
scrapely runs ls janedoe~my-actor --desc --limit 10

Without an Actor ID, runs ls lists the runs of the Actor in the current folder's config (its actorId, or else its name). Outside a project folder it lists all your runs. Each line shows the run ID, the status, and the exit code if there is one.

Inspect a run

scrapely runs info <runId>
scrapely runs info <runId> --verbose
scrapely runs log <runId>

runs info prints the ID, status, exit code, Dataset ID, and KVS ID. Use the dataset ID with scrapely datasets get-items to read the results.

runs log prints the full log once. When the log has expired under the data-retention policy, it says so. To follow a run as it happens, start it with scrapely call, which streams the log.

Wait for a run

scrapely runs wait <runId>
scrapely runs wait <runId> --timeout 600 --poll-interval 5 --json

runs wait checks the run's status until it reaches SUCCEEDED, FAILED, ABORTED, or TIMED-OUT. It exits with code 0 for SUCCEEDED and 1 for anything else.

Abort, resurrect, and delete

scrapely runs abort <runId>
scrapely runs abort <runId> --force
scrapely runs resurrect <runId>
scrapely runs rm <runId> -y

runs abort asks the run to stop gracefully, so the Actor gets the chance to save its state. -f, --force stops it immediately. --json prints { "data": <run> } after the abort request.

runs resurrect restarts a finished run with the same storages, so it can continue where it stopped. It prints the new status, or { "data": <run> } with --json.

runs rm deletes a run. It asks for confirmation unless you pass -y, --yes.

Run statuses

Builds

Builds are owner only, also for public Actors. On someone else's Actor, these commands fail: 403 for an Actor's builds, 404 for a build ID.

scrapely push already starts a build after every upload. Use builds create to rebuild a version without uploading again.

List and inspect builds

scrapely builds ls
scrapely builds ls janedoe~my-actor --desc
scrapely builds info <buildId>
scrapely builds log <buildId>

Without an Actor ID, builds ls lists the builds of the Actor in the current folder's config. Outside a project folder it lists all your builds. Each line shows the build ID, the build number, and the status.

builds info prints the ID, build number, status, exit code, and start and finish times. --json prints the build object.

Wait for a build

scrapely builds wait <buildId> --timeout 900

builds wait checks the build's status while it is RUNNING and exits with code 0 if the build SUCCEEDED, 1 otherwise. It takes the same -t, --timeout, --poll-interval, and --json options as runs wait. On timeout, --json prints "status": "TIMEOUT".

Start a build

scrapely builds create
scrapely builds create janedoe~my-actor --version 0.1 --tag beta --wait --log

builds create builds one version of the Actor without the Docker cache. Without an Actor ID it uses actorId from the config in the current folder. scrapely actors build is the same command.

Tag builds

A build tag is a name, such as latest or beta, that points to one build of the Actor. A run started without -b uses the latest tag.

scrapely builds add-tag -b <buildId> -t beta
scrapely builds remove-tag -b <buildId> -t beta

add-tag only tags builds that have SUCCEEDED. If the tag already points to another build, it moves to this one. remove-tag fails unless the tag currently points to the build you name.

Delete a build

scrapely builds rm <buildId>
scrapely builds rm <buildId> -y

builds rm asks for confirmation unless you pass -y, --yes. The Actor's current User Build can't be deleted; set a different one first.

Build statuses

What's next

• Storage management: read the datasets and key-value stores your runs produce.
• Actors: how versions, builds, tags, and runs fit together.
• Runs API: the endpoints these commands use.