Skip to content
n8n

Web scraping in n8n with the Scrape.ly node

Run ready-made scrapers from your n8n workflows

The Scrape.ly community node for n8n runs scrapers from the Scrape.ly Store inside your workflows. Pick an Actor, such as a Google Maps, LinkedIn or Amazon scraper, give it its input, and the node waits for the run and hands each result to the next node as an n8n item. There are no selectors to write: the Actor does the scraping in the cloud and returns structured data.

A trigger node starts a workflow when a run finishes, and n8n AI agents can use the node as a tool. The node is open source under the MIT license and published on npm as n8n-nodes-scrapely.

What the n8n node can do

  • Run an Actor with the input you give as JSON, wait for it, and output its results. Pick the Actor from a searchable list, or paste its ID such as username/actor-name.
  • Run a saved task, optionally overriding parts of its input.
  • Get a run: the last run of an Actor, or any run by its ID.
  • Read results: dataset items with a limit and offset, or one key-value store record such as OUTPUT.
  • Start a workflow when a run of an Actor or task succeeds, fails, times out or is aborted, with the run's results included if you want them.
  • Work as an AI agent tool, so an n8n agent can run a scraper itself.

How to scrape a website in n8n

1. Install the verified node

In n8n, open the nodes panel, search for Scrapely, pick the version with the verified badge and click Install. It works on n8n Cloud and self-hosted n8n; self-hosted instances can also install n8n-nodes-scrapely under Settings → Community nodes.

2. Add your API key

Create a Scrapely API credential and paste an API key from the dashboard (Settings → API & Integrations → API keys). A scoped key needs actors:run, runs:read and datasets:read, plus webhooks:write for the trigger.

3. Run a scraper

Add the Scrapely node, choose Actor → Run, pick the Actor and paste its input. Wait for Finish and Return Results are on by default, so the next node gets one item per result. A run that doesn't succeed stops the workflow with its status.

4. React to finished runs

For long or scheduled runs, start a second workflow with the Scrapely Trigger. It registers a webhook in your account when you activate the workflow and removes it when you deactivate it. Turn on Include Results to get the dataset items instead of the event. The n8n docs list every operation and option.

n8n web scraping examples

Google Maps leads into Google Sheets

Run the Google Maps Scraper with a search term and a place, then send the items to n8n's Google Sheets node. Each place arrives as one item with its name, address, phone, website and rating when Maps shows them. The Actor input for 20 dentists in Austin:

{
  "searchStringsArray": ["dentist"],
  "locationQuery": "Austin, TX",
  "maxCrawledPlacesPerSearch": 20
}

Don't need n8n in between? Scrape.ly can also send results straight to Google Sheets.

A website as Markdown for an AI agent or RAG

The Website Content Crawler follows the links under a start URL and saves each page's text and Markdown, without the navigation and boilerplate. Pass the markdown field to an embeddings or vector store node to keep an agent's knowledge base up to date. More in website to Markdown for LLMs.

A scheduled price or stock check with Slack alerts

Start the workflow with n8n's Schedule Trigger, run a Store scraper or the Web Scraper with your own page function, compare the values in an If node, and post changes with the Slack node. For other ways to run scrapers on a timer, see how to schedule web scraping.

Process every finished run

When runs start somewhere else, such as a Scrape.ly schedule, the API or another workflow, a workflow that begins with the Scrapely Trigger and Include Results receives each run's items as it succeeds. Clean them up, enrich them and email a report, or watch for failed runs instead.

Why not just use the HTTP Request node?

For a simple page, n8n's HTTP Request and HTML nodes are enough. The Scrape.ly node earns its place when a site renders with JavaScript, blocks requests or changes often:

HTTP Request + HTML nodesScrape.ly node
A simple static pageWorks well, and costs nothing extra.Works too, but you may not need it.
Content rendered by JavaScriptGets the HTML before scripts run, so the data is often missing.Actors can open the page in headless Chrome, for example the Web Scraper with useChrome on.
Blocked requests (403, 429, bot checks)Every request comes from your n8n server’s IP address.Runs happen in the Scrape.ly cloud and can go through Scrape.ly proxies. See anti-blocking.
The site changes its layoutYou find and fix the selectors in your HTML node.The Actor’s developer maintains it, and each Store page shows its success rate and latest automated checks.
Large crawls and long runsThe work runs inside your n8n instance.The run uses Scrape.ly’s memory and timeout. The node waits up to Max Wait (an hour by default), or a trigger picks up the results later.

Thinking of a Puppeteer or Playwright community node instead? They need Chromium inside your n8n container. Our guide to the n8n Puppeteer node compares that route with running the browser on Scrape.ly.

Give your n8n AI agent a scraper

Connect the Scrapely node to an n8n AI Agent as a tool, and the agent can run a scraper and read its results while it works: look up a company's Google Maps listing, read a page it was given, or collect job posts. Agents outside n8n, such as Claude or Cursor, can use the same Actors through the hosted MCP server. See Scrape.ly for AI agents and AI web scraping.

Scrapers to start with

Any Actor you can run on Scrape.ly works in n8n: the scrapers in the Store, your own Actors and your saved tasks. Common starting points:

Prefer plain HTTP? The same Actors run from the Scrape.ly API. For more ways to build lead lists, see lead generation scraping.

More ways to connect Scrape.ly

n8n web scraping FAQ

How can I scrape websites using n8n?

Install the Scrape.ly community node, add your API key, pick a scraper from the Store or one of your own Actors, and the node returns the results as n8n items. For simple static pages, n8n’s HTTP Request and HTML nodes can also fetch a page and pull values out of it.

Is there a Scrape.ly node for n8n?

Yes: n8n-nodes-scrapely, a verified community node, published on npm and open source on GitHub under the MIT license. Install it from the nodes panel by searching for Scrapely, on n8n Cloud or self-hosted n8n. It adds the Scrapely node, which runs Actors and saved tasks and reads runs, datasets and key-value store records, and the Scrapely Trigger.

Does the Scrape.ly node work on n8n Cloud?

Yes. n8n verified the Scrapely node, so an instance owner or admin can install it from the nodes panel on n8n Cloud as well as on self-hosted n8n: search for Scrapely, pick the version with the verified badge and click Install.

Which scrapers can I run from n8n?

Any Actor you can run with your Scrape.ly account: the ready-made scrapers in the Store, your own Actors, and your saved tasks. The Actor’s input goes into the node as JSON.

Can n8n scrape sites that block requests or need JavaScript?

Not reliably with the HTTP Request node alone: it gets the HTML before scripts run, and every request comes from your server’s IP address. A Scrape.ly Actor runs in the cloud, can open pages in headless Chrome and can use Scrape.ly proxies, and the node passes its results back to your workflow.

How do I get scraped data into the next n8n node?

Leave Return Results on: the Run operation then outputs the run’s dataset items, one n8n item each. You can also read a dataset later with Dataset → Get Items, using a limit and offset, or turn on Include Results in the Scrapely Trigger.

Can I start a workflow when a scrape finishes?

Yes, with the Scrapely Trigger. Pick an Actor or saved task and the events (succeeded, failed, timed out or aborted). It registers a webhook in your Scrape.ly account when you activate the workflow and removes it when you deactivate it. This is also the way to handle runs longer than the node’s Max Wait, which is an hour by default.

Can n8n AI agents use Scrape.ly?

Yes. n8n AI agents can use the Scrapely node as a tool, so an agent can run a scraper and read its results. Agents outside n8n can connect to the Scrape.ly hosted MCP server instead.

Is there a free way to scrape websites in n8n?

n8n’s HTTP Request node is free for simple pages. The Scrape.ly node is free too, and runs are billed to your Scrape.ly account at each Actor’s price, like any other run. The Free plan includes $5 of usage every month.

Is web scraping legal?

Collecting publicly available web pages with a program is not illegal in itself in the US, the EU or the UK. What can make it unlawful is what you collect and how: personal data, copyrighted content you republish, pages behind a login, terms you agreed to, and technical barriers you get around. This is general information, not legal advice.

Is web scraping legal?

Sign up at scrape.ly

Create an account to run Actors and connect them.

Get started

The internet is full of data.
Start owning yours.

Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes.

No credit card. No setup calls.