Results & exports

Every run stores its results in Scrape.ly storage. Most Actors write result items to the run's default dataset, and some also write files or a summary to its default key-value store. This page shows how to view those results in the dashboard, export them in different formats, download them over the API, and how long they are kept.

Where a run's results live

Each run gets its own default storages, created when the run starts:

The Actor's README usually says which of these holds its results. See Storage for how the storages work.

View results in the dashboard

Open the run, then the Storage tab, and choose Dataset, Key-value store, or Request queue. The dataset has four views:

• Overview: the item count and the number of fields.
• All fields: every field name found in the items.
• Table: the items as a sortable table.
• JSON: the raw items.

If the Actor defines an output schema, the run also has an Output tab with links to its main results.

All your storages, including those of older runs, are under Storage in the side menu, in the Datasets, Key-value stores, and Request queues tabs. From a storage's page you can Export (datasets), Download records (key-value stores), Rename, or Delete it.

Export a dataset

Click Export on the run page or the dataset page to open Export dataset:

Select fields, Omit fields, and Unwind fields are multi-select lists of the dataset's top-level fields, found by sampling its first and last 100 items. Type to filter the list. To use a field the sample missed, type its name and press Enter or ,. Unwind fields lists the fields that hold arrays or objects.

Download saves the file. View in new tab opens the result in the browser, limited to the first 1,000 items unless you set a limit.

PDF exports

A PDF is a printable report of the items, with the same field options as the other formats.

Pick fields with Select fields to keep wide datasets in table form. For complete exports of large datasets, use CSV, Excel, or JSON.

Download with the API

The first form takes a dataset ID or username~dataset-name. The other three are shortcuts to a run's default dataset: a specific run, your latest run of an Actor, or your latest run of a task. The runs/last forms accept status=SUCCEEDED to skip runs that are still going or failed.

curl "https://api.scrape.ly/v2/datasets/<datasetId>/items?format=csv&fields=title,url,price&clean=true" \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -o results.csv

Item parameters

Responses include the headers X-Apify-Pagination-Offset, -Limit, -Count, -Total, and -Desc, so you can page through large datasets. Very large downloads are streamed. If you start too many large downloads at once, you get 429 with a Retry-After header.

With the clients, use client.dataset(id).listItems() in JavaScript or client.dataset(id).list_items() in Python. See the Datasets API reference for the full response format.

OUTPUT and other key-value records

Many Actors write a summary or a single result to the OUTPUT record of their default key-value store. Others save files there, such as screenshots, PDFs, or HTML.

• keys lists the record keys. It accepts prefix, exclusiveStartKey, and limit (up to 1000).
• records/{key} returns the record with its original content type. Add attachment=true to download it as a file.
• records (without a key) downloads all records as a ZIP file. Filter it with prefix.
• The actor-runs/{runId}/key-value-store/... form reads the run's default store without looking up its ID.

POST /v2/actors/{actorId}/run-sync starts a run and returns its OUTPUT record in one call. See Run synchronously.

The run's own input is the INPUT record, also available at GET /v2/actor-runs/{runId}/input.

Retention

To keep a run's results past the retention window, export them, or copy the items into a named dataset that you create under Storage or with POST /v2/datasets?name=<name>. Renaming a run's default storage does not change its expiry date. Named storages are permanent, but they still count toward storage costs.

Storage is billed by the hour while it exists. If your balance drops to zero, unnamed storages are deleted 3 days later and named storages after a longer grace period. See Storage retention.

Share results

Storages are private to your account: without your token, GET /v2/datasets/{id}/items returns 404. To share results without sharing your token, create a signed URL. Signed URLs always expire.

import { ScrapelyClient } from 'scrapely-client';

const client = new ScrapelyClient({ token: process.env.SCRAPELY_TOKEN, baseUrl: 'https://api.scrape.ly' });

// Valid for 24 hours; defaults to 1 hour
const url = await client.dataset('<datasetId>').createItemsPublicUrl({
    clean: true,
    expiresInSecs: 24 * 60 * 60,
});

Anyone with the URL can download the items until it expires, without a token. They can add item parameters such as &format=csv to it. Signed URLs can only be created for storages you own.

For pipelines, the usual alternative is a webhook that fires when the run succeeds and fetches the items with your token. See Integrations.

What's next

• Integrations: get notified and chain runs
• Storage: datasets, key-value stores, and request queues in depth
• Datasets API and Key-value stores API