Skip to content
Integrations

Azure Blob Storage

Upload the results of each successful run to a container in your Azure storage account.

Uploads the dataset of each successful run as one file to a container in your own Azure storage account. Scrape.ly uses a container SAS URL you create: it only reaches that container, with the permissions and expiry date you choose, and it is stored encrypted and never shown again.

Create a SAS URL

In the Azure portal:

  1. Open the storage account, then Data storage → Containers, and open the container (create one if needed).
  2. Under Settings, click Shared access tokens.
  3. Under Permissions, select only Create and Write. Set an Expiry date, and keep Allowed protocols on HTTPS only.
  4. Click Generate SAS token and URL and copy the Blob SAS URL. It looks like https://<account>.blob.core.windows.net/<container>?sv=…&sig=….

With the Azure CLI, the same link is https://<account>.blob.core.windows.net/<container>? followed by the token this prints:

.bash
az storage container generate-sas --account-name <account> --name <container> \
  --permissions cw --expiry 2027-01-31 --https-only -o tsv

This signs the SAS with the account key, so it can last as long as you set. A SAS signed with your sign-in (--auth-mode login --as-user) lasts at most 7 days. The storage account must allow access from the internet: under Networking, public network access must be enabled for all networks.

Upload results after each run

Set it up with the Upload results to Azure Blob recipe in New automation, or add Azure Blob as a delivery in the automation editor to upload the results of several Actors, after filters and an AI step.

  1. Click New automation (in Automations, or on an Actor's Automations tab) and pick Upload results to Azure Blob, or click Or create manually and, under Send, click Azure Blob.
  2. Paste the Container SAS URL.
  3. Optionally set a Folder, the File format (JSON, JSONL or CSV), and Max items per run (empty uploads every item).
  4. Use Save and send test. A scrapely-test.txt file appears in the folder.

File names: <folder>/<actor-name>/<run-id>.<format>, for example exports/my-actor/a1b2c3d4.csv. Each run gets its own file. The file has the same content as downloading the dataset in that format, except that CSV files have no byte-order mark; CSV cells that start with =, +, - or @ are prefixed with ' so spreadsheets don't run them as formulas.

When it uploads: only for succeeded runs, and only when the run returned items. Large datasets are uploaded in blocks of 8 MB; if an upload fails partway, Azure discards the uploaded blocks on its own after 7 days. An upload that takes longer than 8 minutes is stopped; for very large runs, set Max items per run.

When the link expires: uploads fail with a message saying so. Create a new SAS URL and paste it on the automation's page; leaving the field empty keeps the saved one.

Limits: each Azure Blob automation of this kind uploads at most 30 times an hour, and an account at most 500 exports a day (Google Sheets, S3, GitHub, Airtable, HubSpot, Salesforce, Notion, Dropbox, OneDrive, Postgres, Azure Blob, BigQuery, OpenAI Vector Store, and Snowflake together). If the container doesn't exist, the link has expired, or it may not write, the delivery history on the automation's page says so.

API

A webhook with an azure notification. Only ACTOR.RUN.SUCCEEDED is allowed.

.bash
curl -X POST https://api.scrape.ly/v2/webhooks \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "eventTypes": ["ACTOR.RUN.SUCCEEDED"],
    "condition": { "actorId": "YOUR_ACTOR_ID" },
    "description": "Results to Azure",
    "notification": {
      "channel": "azure",
      "sasUrl": "https://<account>.blob.core.windows.net/<container>?sv=…&sig=…",
      "folder": "exports",
      "format": "jsonl"
    }
  }'

maxItems is optional; leave it out or set it to null to upload every item. Responses carry notification.azureSettings with the folder, format, maxItems, the link's expiresAt, and a sasHint (account/container and the expiry date); the SAS URL is never returned. On PUT, leaving out sasUrl keeps the saved one. In an automation, use a delivery with "channel": "azure" and the same fields.