Skip to content

Dataset Change Tracker

Try for free

Compare every scraper run with the previous one and get only new, changed and removed items, with before → after values per field and % change for numbers. Works with any dataset, JSON or CSV. Alerts via Telegram, Slack, Discord or webhook.

🔄 Dataset Change Tracker

Run your scraper on a schedule and get only what changed since the last run: new items, changed items with a before → after value for every changed field, and removed items. Works with any dataset, JSON file or CSV file, compares 10,000 items in about 7 seconds, needs no code, and can alert you on Telegram, Slack, Discord or any webhook. You pay $1.70 per 1,000 reported changes plus $0.0085 for each started block of 1,000 items compared, with platform usage included.

✨ Why use Dataset Change Tracker?

  • 🆕 New, changed and removed items: new products, jobs or listings, price and stock changes, and items that were sold out or delisted, each as one row.
  • ✏️ Field-level diffs: price: 18.99 → 15.99 (-15.8%), availability: In stock → Out of stock, with delta and percentChange for every number.
  • 🎯 You decide what counts as a change: match items by url, sku or any nested field, watch only price and stock, or ignore fields such as scrapedAt that change on every run.
  • 🧠 Remembers the last run for you: every monitor name keeps a compact, compressed snapshot in your own storage, so each run compares with the previous one automatically. Or compare two versions directly.
  • 🔔 Alerts only when something changed: Telegram, Slack, Discord or a webhook for Make, Zapier, n8n or your own server.
  • 🛡️ Safe by default: an empty result (often a blocked scrape) never reports everything as removed, and a run that hits your cost limit keeps the old snapshot, so no change is lost.

🎯 Use cases

  • 💸 Price monitoring: get the products whose price dropped or rose since yesterday, with the % change.
  • 📦 Stock and availability: catch items that went out of stock, came back, or were delisted.
  • 💼 New job postings or listings: run a jobs, property or car scraper daily and get only the new ads.
  • ⭐ New reviews: keep only the reviews that appeared since the last run.
  • 🏪 Competitor catalogs: see which products a competitor added, removed or renamed.
  • 🔍 SEO and content tracking: follow title, meta description or ranking changes on a list of pages.

🚀 How to use

  1. Pick the dataset of your scraper's latest run in Dataset to check (or paste a JSON / CSV link, or items as JSON).
  2. Set Unique key fields to a field that identifies an item, such as url, id or sku, and add fields that change on every run (scrapedAt, position) to Fields to ignore.
  3. Give the monitor a name, for example laptops-de, and click Start. The first run saves a baseline.
  4. Run it again after the next scrape with the same monitor name: you get only the new, changed and removed items.
  5. To make it automatic, open Automations, create one that runs after your scraper's run finishes, add this actor as a step and set its input to {"datasetId": "{{resource.defaultDatasetId}}", "monitorName": "laptops-de", "idFields": ["url"]}.
  6. Download the changes as JSON, CSV or Excel, or read them through the API.

To compare two versions directly (for example two runs of a scraper), fill Dataset to check with the newer one and Previous dataset with the older one. The form's example does exactly that with a small book shop: yesterday and today.

📥 Input

FieldWhat it does
datasetIdThe latest data: a dataset ID (from the dataset picker, a run's Storage tab or defaultDatasetId).
dataUrl…or a public JSON, JSON Lines or CSV link (dataset export link, Google Sheet published as CSV).
items…or the items as a JSON array.
monitorNameEach name keeps its own snapshot; runs with the same name are compared.
idFieldsField(s) that identify an item, e.g. url, sku, offer.id. Empty: the whole item is the key.
compareFieldsOnly these fields decide whether an item changed. Empty: all fields.
ignoreFieldsFields that are not real changes, e.g. scrapedAt.
reportNew, reportChanged, reportRemovedWhich kinds of change to report (all on by default).
previousDatasetId, previousDataUrl, previousItemsCompare with this version instead of the saved snapshot.
telegramBotToken + telegramChatId, slackWebhookUrl, discordWebhookUrl, webhookUrlOptional alerts.
{
  "datasetId": "4f966164-377d-46ce-9746-f317e212b76b",
  "monitorName": "laptops-de",
  "idFields": ["url"],
  "compareFields": ["price", "inStock", "title"],
  "reportRemoved": true,
  "slackWebhookUrl": "https://hooks.slack.com/services/T000/B000/XXXX"
}

📤 Output

Every change is one row:

FieldDescription
changeTypenew, changed or removed
keyThe item's key (its unique key field values)
changeSummaryThe change in one line, e.g. price: 18.99 → 15.99 (-15.8%)
changedFieldsNames of the changed fields (changed items)
changesPer changed field: field, before, after, kind (modified, added, removed), plus delta and percentChange for numbers
totalChangedFieldsNumber of changed fields
itemThe current item; for removed items, its last known values
monitorNameThe monitor the run belongs to
detectedAtWhen the change was detected (UTC)

Example rows from the form's example run:

[
  {
    "changeType": "changed",
    "key": "https://books.example.com/p/quiet-garden",
    "changedFields": ["price"],
    "changes": [
      { "field": "price", "before": 18.99, "after": 15.99, "kind": "modified", "delta": -3, "percentChange": -15.8 }
    ],
    "totalChangedFields": 1,
    "changeSummary": "price: 18.99 → 15.99 (-15.8%)",
    "item": {
      "url": "https://books.example.com/p/quiet-garden",
      "title": "The Quiet Garden",
      "price": 15.99,
      "availability": "In stock",
      "rating": 4.5,
      "scrapedAt": "2026-11-13T08:00:00Z"
    },
    "monitorName": null,
    "detectedAt": "2026-11-13T04:25:48.028Z"
  },
  {
    "changeType": "removed",
    "key": "https://books.example.com/p/city-of-bridges",
    "changedFields": [],
    "changeSummary": "item no longer present",
    "item": {
      "availability": "Only 2 left",
      "price": 15.0,
      "rating": 4.1,
      "title": "City of Bridges",
      "url": "https://books.example.com/p/city-of-bridges"
    },
    "monitorName": null,
    "detectedAt": "2026-11-13T04:25:48.028Z"
  }
]

The Field changes view of the dataset lists one row per changed field. The run's OUTPUT record holds a summary: counts of new, changed, removed and unchanged items, items compared, skipped duplicates and items without a key, whether the snapshot was updated, and the result of each notification.

Notification messages look like this:

🔔 laptops-de: 3 new, 2 changed, 1 removed (1,240 items checked)
🆕 NEW: https://shop.example.com/p/981
✏️ CHANGED: https://shop.example.com/p/204
    price: 999 → 899
❌ REMOVED: https://shop.example.com/p/377
…and 2 more
Full results: <link to the run's dataset>

The webhook receives a POST with JSON: event (dataset.changes or dataset.no-changes), monitorName, summary, message, resultsUrl, runId, datasetId and the first webhookMaxItems changes (100 by default).

💰 Pricing

You pay for what the run reports and compares. Platform usage (compute, storage) is included.

EventFreeStarterScaleBusiness
Change detected (new, changed or removed item)$0.0017$0.001615$0.00153$0.001445
1,000 items compared$0.0085$0.0085$0.0085$0.0085

"1,000 items compared" is charged once per started 1,000 items read from the latest data.

  • A daily check of 5,000 products where 40 changed: 5 × $0.0085 + 40 × $0.0017 ≈ $0.11 on the Free plan.
  • A run where nothing changed costs only the comparison: 1,000 items ≈ $0.0085.
  • Change types you switch off are not reported and not charged.

❓ FAQ

Why did the first run report nothing? The first run of a monitor saves the baseline. Turn on Report all items as new on the first run if you want everything reported, or compare with a Previous dataset right away.

Every item shows as changed. A field changes on every run, for example a timestamp, a rank or a session ID. Add it to Fields to ignore, or list only the fields you care about in Fields to watch.

Every item shows as new. The key is not stable. Pick a field that stays the same between runs, such as a product ID or a canonical URL.

Can I track several scrapers? Yes. Use a different Monitor name for each one; each name keeps its own snapshot. Without a name, the source actor of the dataset is used.

Where is the history stored, and how do I start over? In a named key-value store in your own account (dataset-change-tracker-state by default), one snapshot per monitor name. Turn on Start over for one run to save a new baseline. Changing the key, watched or ignored fields starts a new baseline automatically, because the old and new snapshots are no longer comparable.

Can I schedule it or use the API? Yes. Schedule it, run it from an automation after your scraper, or call it through the API, the CLI or MCP with the same monitor name each time. Set a maximum cost per run to cap spending: if the run reaches it, it stops cleanly and keeps the previous snapshot.

⚠️ Limits

  • Nested objects are compared field by field up to 4 levels deep; deeper objects and lists show as one changed field.
  • Values longer than 300 characters are kept as a fingerprint plus a 200-character preview, so before shows the preview. Changes are still detected exactly.
  • The snapshot is held in memory while comparing. In our tests 100,000 product rows (10 fields each) peaked at about 185 MB, inside the default 256 MB; for more items, give the run more memory or turn off Show which fields changed to keep only fingerprints.
  • JSON and CSV links are downloaded whole; use a dataset ID for very large data.
  • When an item's key appears twice in one version, only the first one is used; items without a key value are skipped. Both are counted in the summary.
  • With Max items to compare, removed items are not reported for that run, because unread items may still exist.
  • Two runs of the same monitor at the same time each save their own snapshot; the one that finishes last is kept. Use one monitor name per schedule.
  • Datasets are read through your own account, so only datasets you own (or that a run of yours created) can be checked. There is no email field: for email, add an Email me the results automation to this actor.

Related actors

//From the blog\\

Latest articles

All articles
G2
4.7/5
Excellent
Trustpilot

The internet is full of data.
Start owning yours.

Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes.

Get Started FreeChat with Support
No credit card. No setup calls.