Web Scraping Ecommerce Sites: Product Data From Amazon, Walmart, Shopify and More

Web scraping ecommerce sites means collecting product pages and search results from online stores and marketplaces as structured rows: title, price, previous price, seller, stock, rating, variants and identifiers. The fastest way to do it is a ready-made scraper per store, so you choose the stores and the products, not a parsing framework, and get the same JSON, CSV or Excel output from each.

This guide covers what each major store gives you and what it costs per 1,000 products, what teams use the data for, a real cross-store check we ran on 26 October 2026 (USB-C cables on Amazon, Walmart and AliExpress), a Python script that turns the three into one comparable table, and the pitfalls that make or break product data. For Google Shopping's cross-merchant prices, see how to scrape Google Shopping.

What ecommerce data you can scrape

Each store has its own Actor in the Scrape.ly Store. Prices are per product saved, with platform usage included, checked on the Store pages on 26 October 2026:

The Amazon Actor adds optional paid extras (other sellers' offers, seller ratings, a fixed delivery ZIP) and a $0.0004 start fee per run; Google Shopping has a $0.00005 start fee. For full AliExpress product pages with every variant and shipping option, the AliExpress Product Scraper costs $0.41 per 1,000. The Store also covers Best Buy, Target, Home Depot, Etsy, idealo and Keepa price history, among others.

What teams use ecommerce data for

• Competitor price monitoring. The same products at rival stores every day, with who moved and by how much. Our price scraping guide covers this in depth.
• Assortment and catalog research. Which products, variants and price points a competitor carries, and which you don't.
• MAP and brand protection. Brands check which marketplace sellers advertise below the minimum price, and which listings use their name.
• Sourcing and dropshipping. Compare a supplier's AliExpress price, shipping and stock with what the same product sells for on Amazon or Walmart.
• Demand signals. "Bought in past month" on Amazon, order counts on AliExpress and sold counts on eBay show what actually sells.
• Retail media. Sponsored flags and search positions show who pays for visibility on a keyword.

A real cross-store check: USB-C cables on three marketplaces

On 26 October 2026 we searched for USB-C cables in three stores, five products each:

Our account was billed $0.00456, $0.00868 and $0.02452 after its plan's Store discount. The Walmart run's log shows how little friction there was:

2026-10-26T14:33:31.152Z [Scrapely] Container started (setup 0.9 s)
2026-10-26T14:33:32.542Z [scrapely.proxy_configuration] INFO  Platform proxy access check succeeded.
2026-10-26T14:33:32.542Z [scrapely.actor] INFO  Scraping 1 input(s), up to 5 results, search results only, network: datacenter.
2026-10-26T14:33:36.502Z [scrapely.actor] INFO  Done. Saved 5 results. Requests: 1, bot checks: 0, browser pages: 0, proxy: datacenter.
2026-10-26T14:33:36.705Z [scrapely.actor] INFO  [Status message]: Saved 5 results.
2026-10-26T14:33:37.189Z [Scrapely] Container state: status=exited exitCode=0 oomKilled=false

Sticker prices alone would rank these 15 cables almost at random, because half of them are multi-packs. So the next step is normalization.

Web scraping ecommerce sites with Python

This script (pip install scrapely-client-python) runs the three searches above, each capped at 5 cents, reads the pack size from the title and sorts every row by price per cable:

import os
import re
from decimal import Decimal

from scrapely_client import ScrapelyClient

client = ScrapelyClient(os.environ["SCRAPELY_TOKEN"], api_url="https://api.scrape.ly")

def items(actor, run_input):
    run = client.actor(actor).call(run_input=run_input, max_total_charge_usd=Decimal("0.05"))
    return list(client.dataset(run.default_dataset_id).iterate_items())

def pack_size(title):
    m = re.search(r"(\d+)\s*-?\s*(?:pack|pcs|pieces|pc)\b", title, re.I)
    return int(m.group(1)) if m else 1

rows = []
for p in items("shelfbright/amazon-product-scraper", {
        "categoryOrProductUrls": [{"url": "https://www.amazon.com/s?k=usb+c+cable"}],
        "maxItemsPerStartUrl": 5, "scrapeProductDetails": False}):
    if (p.get("price") or {}).get("value"):
        rows.append(("Amazon", p["title"], p["price"]["value"]))
for p in items("shelfbright/walmart-scraper", {
        "search": "usb c cable", "minPrice": 10, "maxPrice": 20,
        "scrapeProductDetails": False, "maxItems": 5}):
    rows.append(("Walmart", p["name"], p["price"]))
for p in items("shelfbright/aliexpress-search-scraper", {
        "keywords": ["usb cable"], "maxItems": 5, "minPrice": 10, "maxPrice": 20}):
    rows.append(("AliExpress", p["productTitle"], p["price"]))

for store, title, price in sorted(rows, key=lambda r: r[2] / pack_size(r[1])):
    n = pack_size(title)
    print(f"{store:10}  ${price:>6.2f}  x{n}  ${price / n:5.2f} per cable  {title[:50]}")

Applied to the 15 rows our three runs returned, the normalization and sort print:

AliExpress  $ 11.85  x8  $ 1.48 per cable  8pcs 100W 3 In 1 Fast Charging Cable 1.2m/4ft Nylo
Amazon      $ 13.99  x5  $ 2.80 per cable  CLEEFUN USB C to USB C Cable 6ft, 5-Pack C to C Ca
Walmart     $ 10.90  x3  $ 3.63 per cable  Aioneus USB C Cable 10FT 3Pack, Braided Type C Cha
AliExpress  $ 13.33  x3  $ 4.44 per cable  3PCS Fast Charging Cable 120W USB A to Type C & 10
AliExpress  $ 13.93  x3  $ 4.64 per cable  CABLETIME 3 Pack 240W USB C to USB C Cable Type C 
Amazon      $  9.99  x2  $ 5.00 per cable  Anker USB C to USB C Cable, 60W Fast Charging Cabl
Amazon      $  9.99  x2  $ 5.00 per cable  Anker USB C to USB C Cable, 60W Fast Charging Cabl
AliExpress  $ 11.52  x2  $ 5.76 per cable  AUFU 2-Pack USB C to USB C Cable 240W C to C Cable
Amazon      $  8.97  x1  $ 8.97 per cable  LISEN USB C to USB C Cable, 240W Fast Charging Typ
AliExpress  $ 14.42  x1  $14.42 per cable  Baseus 3 in 2 USB Type C Cable for iPhone 15 14 13
Walmart     $ 14.88  x1  $14.88 per cable  Belkin Braided USB-C to USB-C Cable ( 10ft, 60W) -
Walmart     $ 14.99  x1  $14.99 per cable  Apple 60W USB-C Charge Cable (1m) Fast Charge Cabl
Walmart     $ 16.99  x1  $16.99 per cable  Verbatim 240W USB4 USB-C Cable with DuPont Kevlar 
Walmart     $ 19.99  x1  $19.99 per cable  RVP+ USB4 Thunderbolt 4 Cable 240w, 40Gbps 8K Vide
Amazon      $ 24.00  x1  $24.00 per cable  Apple 240W USB-C to USB-C Woven Charge Cable (2 m)

What the table teaches, in order of importance:

• Normalize before you compare. Per cable, the cheapest row was an AliExpress 8-pack at $1.48, then an Amazon 5-pack at $2.80 and a Walmart 3-pack at $3.63. Single cables ran from $8.97 to $24.00. The cheapest sticker price ($8.97) was ninth per unit.
• Then compare like with like. The $1.48 cable is a 100W 3-in-1 multi-connector; the $19.99 one is a 40 Gbps USB4 Thunderbolt cable. Pack size is the first filter; wattage, length and data speed come next. Match on identifiers (ASIN, UPC, model number) where the store gives them.
• Sponsored isn't organic. Three of the five Walmart rows were sponsored (isSponsored: true), all from third-party sellers; the two non-sponsored rows were sold by Walmart.com itself. None of the five Amazon search rows was sponsored. Keep the flag if you track search rank.
• Demand travels with price. Four of the five Amazon rows showed "10K+ bought in past month"; the AliExpress cards ranged from 12 to 1,137 orders. That's the sales signal a price alone doesn't give.

The same runs work over the REST API, for example POST https://api.scrape.ly/v2/actors/shelfbright~walmart-scraper/run-sync-get-dataset-items with the JSON input (all three runs finished in under 7 seconds), and from AI agents: add https://mcp.scrape.ly to Claude Code or Cursor and ask "Use scrapely to search Amazon and Walmart for usb c cable and give me a table sorted by price per cable." See the Amazon MCP guide for agent setup.

How to scrape an ecommerce website without code

1. Pick the stores. Open each Actor and set the input: Amazon search or product URLs, a Walmart keyword with a price range, a Shopify collection URL. Use the Store presets as a starting point.
2. Save each input with Save as new… in the Input menu.
3. Schedule. In Automations, create one automation On a schedule (daily at 9:00 is the default) and add up to 10 Actors under Run.
4. Filter. Keep Only new items since the last run, or Only rows where… price is less than a threshold.
5. Deliver. Append to Google Sheets, post to Slack or email, call a webhook, or upload each run to S3 or R2.

Shopify stores are the easiest place to start: one URL covers a whole collection. In our Shopify guide, a run from 23 October 2026 saved 10 Allbirds men's shoes with 130 variants in 6 seconds, including one colorway marked down from $140 to $99 and one sold out in every size.

What it costs

Fixed prices per product, platform usage included, and a maximum charge per run that stops the run when reached. Our three runs above cost $0.0472 together at list price.

A worked example: checking 500 products daily on Amazon ($1.06 per 1,000), Walmart ($2.17) and a Shopify competitor ($0.53) costs 500 × ($0.00106 + $0.00217 + $0.000534) = $1.88 a day, plus $0.0004 per Amazon run, about $56.50 for 30 days. Swap Walmart for Google Shopping ($4.17) when you want many merchants per query. The Free plan's $5 of monthly credit covers about 1,300 product checks across those three stores; Starter at $19 a month includes $19 of credit.

What to watch out for

• Pack sizes and bundles. As above: normalize per unit before you compare, and keep the raw title for checking.
• Search rows vs product pages. On Amazon search rows, three of our five getPriceBeforeDiscount values were lower than the price (the Anker 2-pack at $9.99 showed $5.00), so they aren't a crossed-out price. Open product pages (scrapeProductDetails: true) and read listPrice when discounts matter. On AliExpress, a card shows one variant's price; the product page has the full range.
• Formats differ. Shopify prices are integers in minor units (199500 is $1,995.00), Google Shopping returns text such as "$19.99", Amazon and Walmart return numbers. Store the currency with every price.
• Location changes prices. Amazon prices and delivery depend on the delivery ZIP, AliExpress on shipTo and the proxy location. Keep them fixed across runs.
• Blocking. Big retailers rate-limit bursts. The Actors run on Scrape.ly's proxies and switch to residential addresses or a browser when a page refuses them, so you never manage IPs.
• Sponsored and personalized results. Search positions change by session and include paid placements; flag them and compare like with like.

Is web scraping ecommerce data legal?

Prices and product facts on public pages are information any shopper can see, and the Actors above read public pages without logging in. Store terms still apply. Amazon's Conditions of Use (last updated August 14, 2026, read on 26 October 2026) say its license does not include "any collection and use of any product listings, descriptions, or prices" or "any use of data mining, robots, or similar data gathering and extraction tools". AliExpress's terms are similar; our AliExpress guide quotes them. Whether and how such terms bind you depends on your jurisdiction and your use.

Product photos and descriptions are copyrighted, so don't republish them wholesale. Marketplace seller names can belong to individuals, so collect only what you need. This is not legal advice: if scraped data drives a commercial product or automated repricing, ask a lawyer about your case.

Why Scrape.ly for ecommerce data

• Every major store, one platform. Amazon, Walmart, eBay, Shopify, Google Shopping, AliExpress and more, with the same API, schedules and exports. The Amazon scrapers page lists every Amazon tool, from bestsellers to reviews.
• Pennies per check. $0.53 to $6.13 per 1,000 products with platform usage included, and our 15-product check cost under 5 cents at list price.
• Monitoring built in. Schedules, filters, change detection and delivery to Sheets, Slack, webhooks and S3 come with the platform.
• Ready for agents. The same Actors run from Claude, Cursor or ChatGPT through the hosted MCP server.

FAQ

What is ecommerce scraping?

Ecommerce scraping is the automated collection of product data from online stores: titles, prices, sellers, stock, ratings and variants. Teams use it for price monitoring, catalog research, MAP enforcement and sourcing.

How do I scrape an ecommerce website?

Pick the Actor for the store (Amazon, Walmart, Shopify, eBay, Google Shopping, AliExpress), give it keywords or URLs, run it and export the rows. For a store without its own Actor, Scrape.ly's general crawlers and the hosted browser cover custom sites.

Is there one ecommerce scraper for every store?

Each store has its own Actor, because each page layout and anti-bot setup is different, but they share one platform, one token and one output format. Browse them in the Store.

How do I compare the same product across stores?

Match on identifiers first (ASIN, UPC, GTIN, model number), then normalize price per unit and keep the currency, as the script above does with pack sizes. Google Shopping is a shortcut: one query returns offers from many merchants.

How often should I scrape product prices?

Daily covers most catalogs. Fast-moving categories and promotions may justify several runs a day; filter for changes so you only act on moves.

Can I scrape Shopify stores?

Yes, any public Shopify storefront. Give the Shopify Scraper a store, collection or product URL and it returns every variant with price, previous price and stock status. Password-protected stores are skipped.