Skip to content
Integrations

Airtable

Add the results of each successful run as records to a table.

Adds the dataset items of each successful run as records to an Airtable table. You connect your Airtable account once, and choose which bases Scrape.ly may write to.

Connect Airtable

  1. In the dashboard, open Settings → Connections and click Connect account next to Airtable (or Connect Airtable in the automation form).
  2. Sign in to Airtable and pick the bases Scrape.ly may add records to, then click Grant access.

Scrape.ly can only add records and read the column names of the bases you picked; it can't read your records or change your tables. To use another base later, click Add bases and pick it. You stay connected: the sign-in renews itself, and only expires if no Airtable automation runs for 60 days. Disconnect removes the connection, and Airtable automations stop until you connect again.

Send results to a table

  1. Click New automation and pick Send results to Airtable (on an Actor's Automations tab the Actor is already chosen).
  2. Open the table in Airtable and copy the link from the address bar, such as https://airtable.com/appXXXXXXXXXXXXXX/tblXXXXXXXXXXXXXX/viwXXXXXXXXXXXXXX. Paste it into Table link.
  3. Save and send a test. A Scrapely connected ✓ record appears in the table's first text column.

Columns: Airtable only accepts fields that exist in the table, so a result field is written when the table has a column with the same name (upper or lower case doesn't matter). Nested fields are named with dots, such as price.amount, and arrays are written as JSON text. Add a column for each field you want; fields without a column are skipped, and the delivery history lists them. Airtable converts the values to the column type, so text can go into number, date, or single select columns. A link in an attachment column becomes the attached file. Columns Airtable fills itself, such as formulas, lookups, and autonumbers, are never written.

When it writes: only for succeeded runs, and only when the run returned items. Max records per run (1 to 10,000, default 1,000) caps how many items are added. Each run adds new records; existing records are never changed or removed.

Limits: Airtable takes 10 records per request and 5 requests a second per base, so 1,000 records take about 20 seconds. Each Airtable automation of this kind writes at most 30 times an hour, and an account at most 500 exports a day (Google Sheets, S3, GitHub, and Airtable together). If Airtable isn't connected, the base wasn't picked when connecting, or none of the fields match a column, the delivery history on the automation's page says so.

API

A webhook with an airtable notification. Only ACTOR.RUN.SUCCEEDED is allowed, and Airtable must be connected in the dashboard first.

.bash
curl -X POST https://api.scrape.ly/v2/webhooks \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "eventTypes": ["ACTOR.RUN.SUCCEEDED"],
    "condition": { "actorId": "YOUR_ACTOR_ID" },
    "description": "Results to Airtable",
    "notification": {
      "channel": "airtable",
      "tableUrl": "https://airtable.com/appXXXXXXXXXXXXXX/tblXXXXXXXXXXXXXX",
      "maxRows": 1000
    }
  }'

Responses carry notification.airtableSettings with baseId, tableId, and maxRows, and connected says whether Airtable is still connected. In an automation, use a delivery with "channel": "airtable" and the same tableUrl and maxRows fields.