Price Scraping: How to Monitor Competitor Prices Automatically

Price scraping is the automated collection of product prices from retailer and marketplace pages, so you can compare them, track how they change and react before your customers notice. The fastest way to start is a ready-made price scraper for each store you watch, run on a schedule, with the results landing in a spreadsheet, a bucket or your own system.

This guide explains what price scraping is used for, shows a real cross-retailer check we ran on 22 October 2026 (one keyboard, five stores, five prices), and walks through a daily monitoring workflow on Scrape.ly with no code, then the same comparison in Python. Every price and cost below comes from those runs.

What price scraping is, and what you get

A scraper built for prices opens the same pages a shopper sees (search results, category pages, product pages) and saves the price with the context that makes it comparable: the product, the seller, the condition, the stock, the previous price and the identifiers. On Scrape.ly each store has its own Actor in the Store, so you pick the stores, not a scraping framework.

The Store also has Actors for Best Buy (regular price, savings and open-box prices), Home Depot and idealo, among others. Google Shopping is the shortcut: one query returns offers from many merchants at once.

What businesses use it for

• Competitor price monitoring. Check the same products at your rivals every day and see who moved, by how much, and when.
• MAP monitoring. Brands with a minimum advertised price policy check what resellers advertise and flag listings below the line. Seller and merchant names on every row tell you who to call.
• Repricing. Feed fresh competitor prices into your own pricing rules, so you match or undercut where it pays and hold your price where you can.
• Promotion tracking. wasPrice and getPriceBeforeDiscount show when a store runs a discount and how deep it goes.
• Assortment and stock gaps. Availability and "in stock" fields show which competitor ran out, and which products they carry that you don't.
• Marketplace research. eBay's condition and sold counts separate new from refurbished prices and show which price points actually sell.

A real cross-retailer check: one keyboard, five prices

On 22 October 2026 we ran the Google Shopping Scraper for "logitech mk270 wireless keyboard mouse combo" in the US, 5 products. It finished in 3 seconds and was charged $0.0152. We put its rows next to the Logitech MK270 row from our Amazon run the same day (the Amazon Product Scraper on a keyboard search, 5 products in 6.1 seconds, $0.0046):

Three things stand out, and they are the whole craft of price scraping in one table:

• The same model spans $19.99 to $96.13. Amazon and Logitech's own store sat at $19.99, Walmart at $26.99. The Best Buy row is almost five times the cheapest one. A row like that is a signal to check the listing before you act on it (a bundle, a pack or a marketplace offer would all look like this); the scraper reports what Google shows.
• Titles don't match products. Two of five Google Shopping rows don't name the MK270 at all, and one of them is a different product (a compact Bluetooth combo). Match on identifiers (ASIN, UPC, shoppingId) or at least the model number, never on the title alone.
• Discounts are data. Amazon's row carried both the current price and the pre-discount $29.99, so a single run tells you there is a promotion running, not just a price.

The Google Shopping run's log:

2026-10-22T18:27:02.830Z [Scrapely] Container started (setup 0.8 s)
2026-10-22T18:27:04.337Z [scrapely.actor] INFO  Using fast mode. A search switches to browser mode only if fast mode fails.
2026-10-22T18:27:04.758Z [scrapely.actor] INFO  Saved 5 products: 5 in fast mode (1 result pages), 0 in browser mode; 0 searches switched to browser mode.
2026-10-22T18:27:05.208Z [Scrapely] Container state: status=exited exitCode=0 oomKilled=false

For a category view, the Walmart Scraper's preset "Laptops $300 to $1,000, 10 results" returned 10 products in 7.1 seconds for $0.0204. The cheapest three:

All three are third-party sellers rather than Walmart itself (one ships through Walmart's own fulfillment, per its fulfillmentType), which is exactly what a brand doing MAP checks needs to know. Our Walmart scraping guide has the full run.

How to set up competitor price monitoring without code

Scrape.ly's automations turn one-off runs into a daily price check:

1. Pick the stores. Open each Actor in the Store and set up the input: a Google Shopping query list, Amazon search or product URLs, a Walmart search with price bounds. Use Save as new… in the Input menu to keep it as a saved input.
2. Schedule it. Open Automations in the side menu, click New automation, and set When to On a schedule (daily at 9:00 is the default). Add up to 10 Actors under Run; they start at the same time.
3. Keep what matters. Under Keep, add Only rows where… price is less than your threshold. Number comparisons also read prices written as text, such as Google Shopping's "$19.99". Or use Only new items since the last run with a key field such as url.
4. Let AI spot the moves (optional). The AI step's Compare with last run mode answers "Which prices went down since the last run, and by how much?" in plain text. It is billed per use, like the AI chat, and works best for short lists rather than exact totals over thousands of rows.
5. Send the results. Append rows to Google Sheets, post to Slack, Discord or email, or call your webhook. For a data lake, add the Amazon S3 / R2 integration on the Actor, which uploads every successful run as its own JSON, JSONL or CSV file.

Set Max cost per run in each run's options, and the run stops when it reaches it. That's the guarantee that a broad query never turns into a surprise bill.

Price scraping with Python: compare prices across retailers

Create an API token under Settings → API & Integrations and export it as SCRAPELY_TOKEN. A Google Shopping run of a few products finishes in seconds, so the synchronous endpoint returns the offers directly:

curl -X POST "https://api.scrape.ly/v2/actors/shelfbright~google-shopping-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"queries": ["logitech mk270 wireless keyboard mouse combo"], "maxItems": 5, "countryCode": "us"}'

This Python script (pip install scrapely-client-python) runs Google Shopping and Amazon for the same product, turns every price into a number, keeps the rows that name the model, and flags anything over twice the median:

import os
import re
import statistics
from scrapely_client import ScrapelyClient

client = ScrapelyClient(os.environ["SCRAPELY_TOKEN"], api_url="https://api.scrape.ly")

def run_items(actor, run_input):
    run = client.actor(actor).call(run_input=run_input)
    return list(client.dataset(run.default_dataset_id).iterate_items())

def to_number(price):
    if isinstance(price, (int, float)):
        return float(price)
    match = re.search(r"\d[\d,]*(?:\.\d+)?", price or "")
    return float(match.group(0).replace(",", "")) if match else None

offers = []
for p in run_items("shelfbright/google-shopping-scraper",
                   {"queries": ["logitech mk270 wireless keyboard mouse combo"], "maxItems": 5, "countryCode": "us"}):
    offers.append((p["merchantName"], p["productName"], to_number(p["price"])))
for p in run_items("shelfbright/amazon-product-scraper",
                   {"categoryOrProductUrls": [{"url": "https://www.amazon.com/s?k=logitech+mk270"}], "maxItemsPerStartUrl": 3}):
    offers.append(("Amazon", p["title"], to_number((p.get("price") or {}).get("value"))))

model = [o for o in offers if o[2] is not None and "mk270" in o[1].lower()]
median = statistics.median(price for _, _, price in model)
for merchant, title, price in sorted(model, key=lambda o: o[2]):
    flag = "  <- check listing (over 2x median)" if price > 2 * median else ""
    print(f"{price:8.2f}  {merchant:10}  {title[:48]}{flag}")

Applied to the rows in the table above, the model filter keeps four offers (Amazon, Logitech, Walmart, Best Buy), the median is $23.49, and only the $96.13 Best Buy row gets flagged. In production, replace the title filter with an identifier match and write the rows to your database or a sheet.

AI agents can run the same check over MCP: add https://api.scrape.ly/mcp to Cursor or Claude Code and ask "Use scrapely to check the price of the Logitech MK270 on Google Shopping and Amazon in the US and tell me the cheapest offer."

What it costs

Every Actor charges a fixed fee per result, plus small platform usage. Our runs on 22 October 2026:

• Google Shopping: 5 products × $0.003 = $0.015 in result fees; $0.0152 charged in total (compute added $0.0002).
• Amazon: 5 products × $0.0007 = $0.0035; $0.0046 charged in total.
• Walmart: 10 products × $0.002 = $0.02; $0.0204 charged in total.

A worked example: monitoring 100 products daily on Google Shopping, Amazon and Walmart is 100 × ($0.003 + $0.0007 + $0.002) = $0.57 a day in result fees, about $17.10 for 30 days, before platform usage. The Free plan's $5 of monthly credit covers a smaller pilot; the Starter plan at $19 a month includes $19 of credit.

What to watch out for

• Product matching. As the table shows, titles vary and some rows are different products. Match on ASIN, UPC, GTIN or shoppingId, and keep the model number as a fallback.
• Location changes prices. Amazon prices depend on the delivery location (our row said "Delivering to Chicago 60602"); the Amazon Actor can set a ZIP code, and Google Shopping takes a country and a city. Keep these fixed across runs, or you'll compare two different markets.
• Text versus numbers. Google Shopping returns the price as shown ("$19.99"), Amazon as a number. Normalise before you compare, as the script does, and store the currency.
• Shipping and condition. On eBay, a cheap refurbished item with $10 shipping isn't cheaper than a new one with free shipping. Compare totals and like-for-like conditions.
• Extras cost time. Google Shopping's comparison prices and sponsored listings need browser mode, which takes 30 to 60 seconds for 10 products and adds browser time to the cost. Turn them on only when you need every seller's price.
• Blocking. Retailers block bursts of requests. The Actors run on Scrape.ly's built-in proxies, so you never manage IPs; you only choose the frequency. Daily is enough for most catalogs.

Is price scraping legal?

Prices on public product pages are factual information that any visitor can see, and the Actors above read only public pages without logging in. Store terms still apply. Amazon's Conditions of Use (last updated August 14, 2026), for example, say its license does not include "any collection and use of any product listings, descriptions, or prices" or "any use of data mining, robots, or similar data gathering and extraction tools". Whether and how such terms bind you depends on your jurisdiction and your use.

Price data rarely contains personal data, but seller names on marketplaces can belong to individuals, so collect only what you need. Keep request rates reasonable. This is not legal advice: if scraped prices drive automated repricing or a commercial product, ask a lawyer about your case.

Why Scrape.ly

• Every store is one Actor away. Amazon, Walmart, eBay, Google Shopping, Best Buy, Home Depot, idealo and Keepa price history are ready-made in the Store, so adding a retailer is a new run step, not a new scraper.
• Pennies per check. Fixed per-result prices ($0.70 per 1,000 products on Amazon, $2.00 on Walmart, $3.00 on Google Shopping), a maximum charge per run and a prepaid balance.
• Monitoring is built in. Schedules, price filters, "only new items", an AI compare step and deliveries to Google Sheets, Slack, email, webhooks and S3/R2 come with the platform.
• Ready for agents. The same Actors run from Claude, Cursor or ChatGPT over the hosted MCP server.

FAQ

What is a price scraper?

A price scraper is a program that reads product pages and saves prices as structured data. On Scrape.ly, each store has a ready-made one in the Store that you run from the console, the API or a schedule.

Is there an Amazon price scraper with price history?

Two Actors split the job. The Amazon Product Scraper returns today's price, the pre-discount price and the ASIN from a search, category or product URL; our guide on how to scrape Amazon covers it step by step, including ZIP-code pricing. The Keepa Scraper adds the history: 30/90/180/365-day averages and all-time lows per ASIN.

How often should I scrape competitor prices?

Daily covers most catalogs and keeps costs low. Fast-moving categories such as electronics deals may justify several runs a day; schedule them with a custom cron expression and filter for changes so you only get alerts that matter.

Can I track competitor prices in Google Sheets?

Yes. Add a Google Sheets delivery to your automation, share the sheet with Scrape.ly's service account, and choose Add rows to build a price history or Replace the tab to keep only the latest prices.

Can I compare prices on eBay too?

Yes. The eBay Scraper returns price, shipping cost, condition and sold count per active listing. See how to scrape eBay for a real run with prices from $55.45 to $129.99.