--- title: "6 Best Web Scraping MCP Servers in 2026 | Scrape.ly" description: "The 6 best web scraping MCP servers in 2026: Scrape.ly, Firecrawl, Bright Data, Apify, Playwright and Puppeteer, with tools, sign-in, free tiers and setup." url: https://scrape.ly/blog/best-mcp-servers-for-web-scraping markdown: https://scrape.ly/blog/best-mcp-servers-for-web-scraping.md --- [AI agents](https://scrape.ly/blog/category/ai-agents.md) # 6 Best Web Scraping MCP Servers in 2026 (Setup for Claude Code and Cursor) The 6 best MCP servers for web scraping in 2026, compared on hosting, sign-in, tools, free tiers and spending control, with copy-paste setup for Claude Code, Cursor and VS Code and a real agent run. [Scrape.ly Team The team behind Scrape.ly](https://scrape.ly/blog/author/scrapely-team.md) - Published Oct 23, 2026 - Reading time 10 min read ![Pixel-art robot with an orange >\_ on its terminal head plugs a cable into a server rack labelled MCP with the Scrape.ly logo on its front, while two more cables run into it from web pages on the left](https://scrape.ly/media/best-mcp-servers-for-web-scraping.png?w=1080) The best web scraping MCP servers in 2026 are Scrape.ly, Firecrawl MCP, Bright Data MCP, Apify MCP, Playwright MCP and Puppeteer MCP. Scrape.ly is the web scraping MCP server we recommend first: one OAuth sign-in gives your agent a Store of 568 ready-made scrapers to find, run and read, with a dollar cap on every run. The six split into two kinds. Data servers (Scrape.ly, Firecrawl, Bright Data, Apify) run the scraping on hosted infrastructure and hand back Markdown or JSON. Browser servers (Playwright, Puppeteer) let the agent drive a browser on your machine, one click at a time. Below is how they compare, how to set each one up, and a real run we did through MCP on 22 October 2026. Every competitor fact comes from its official README or docs, read that day. ## What to look for in a web scraping MCP server - **Hosted, so there's nothing to install.** A remote server over streamable HTTP works the same in Claude, Cursor, VS Code and ChatGPT. - **OAuth sign-in.** The client opens a browser window and you approve it. No keys pasted into config files or URLs. - **Tools for real sites, not just "fetch a URL".** Google Maps places, Amazon products and job listings need site-specific scrapers that return clean fields. - **Results the model can page through.** Datasets with offsets and field selection, instead of one giant blob in the context window. - **Spending control.** An agent that runs paid jobs needs a cap per run and a daily limit on its key. - **Anti-bot handling and proxies** built in, so the agent doesn't fail on the first protected page. ## Web scraping MCP servers compared | Server | Hosted endpoint | Sign-in | What the agent gets | Free tier | | --------------- | ------------------------------------------- | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ------------------------------------------------------- | | **Scrape.ly** | `https://api.scrape.ly/mcp` | OAuth 2.1, Bearer token or scoped API key; x402 for agents without an account | Search, inspect and run any of 568 Store Actors; runs, datasets, tasks, automations, docs | $5 of usage every month, no card | | Firecrawl MCP | `https://mcp.firecrawl.dev/v2/mcp` | OAuth, API key, or keyless for scrape, search and parse | 27 tools: scrape, map, crawl, search, interact, agent, monitor and more | 1,000 credits a month, plus a rate-limited keyless tier | | Bright Data MCP | `https://mcp.brightdata.com/mcp?token=…` | API token in the URL | 32 tools: search, Markdown scraping, 1,200+ pre-built scrapers, 14 browser tools | 5,000 requests a month | | Apify MCP | `https://mcp.apify.com` | OAuth or Bearer token | Search, inspect and run Apify Store Actors; runs, storage, tasks, schedules, docs | $5 of usage every month | | Playwright MCP | Runs locally (`npx @playwright/mcp@latest`) | None: your machine | Browser control through accessibility snapshots: navigate, click, type, fill forms, tabs | Free to run | | Puppeteer MCP | Runs locally (npx or Docker) | None: your machine | 7 tools: navigate, screenshot, click, hover, fill, select, evaluate | Free to run (MIT) | ## 1. Scrape.ly MCP: 568 scrapers behind one sign-in The [Scrape.ly MCP server](https://scrape.ly/mcp.md) is hosted at `https://api.scrape.ly/mcp` and speaks the MCP streamable HTTP transport. Clients that support MCP authorization find the OAuth flow on their own, register themselves and open a browser window where you approve access. Under the hood it's OAuth 2.1 with PKCE and dynamic client registration; access tokens last 24 hours and refresh tokens renew for 365 days from their last use, so you stay signed in. Clients that can't do OAuth send a Bearer token instead. The agent works through a small set of tools: | Tool | What it does | | --------------------- | --------------------------------------------------------------------------------------------------- | | `search-actors` | Searches the Store by keywords and returns up to 10 Actors | | `fetch-actor-details` | Returns an Actor's description, input schema, README, pricing and output schema | | `call-actor` | Runs any Actor with a JSON input, waits up to 45 seconds and returns the run status and storage IDs | | `get-actor-run` | Reports a run's status and stats, waiting up to 45 seconds for it to finish | | `get-actor-output` | Reads a page of the run's dataset items (about 50 KB per page) with field selection | Runs, storage, tasks, automations and docs search are enabled by default too. That small surface is the point: instead of one tool per site, the agent finds the right scraper in the [Store](https://scrape.ly/store.md), reads its input schema and runs it. Google Maps, Amazon, LinkedIn Jobs, TikTok, Zillow, Google Search, news articles and whole-site crawls all go through the same three calls. If you want specific Actors as their own tools, add `?actors=owner/name` to the URL (up to 50). What makes it safe to hand to an autonomous agent: - **A dollar cap per run.** `call-actor` takes `callOptions.maxTotalChargeUsd`, which caps everything the run can cost, and `maxItems` for pay-per-result Actors. - **Scoped API keys with a daily charge limit.** Give the agent a key with only `actors:run` and `runs:read`, or the read-only preset, under **Settings → API keys**. A key's own max charge per run caps every run it starts ([API keys docs](https://scrape.ly/docs/api-keys.md)). - **Pay per call without an account.** Agents with a USDC wallet on Base can pay through x402 instead of a token. - **Results that fit the context.** `get-actor-output` pages through datasets, cuts long text values and tells the agent `hasMore` and `nextOffset`, so a 10,000-row dataset never lands in the context in one go. There's also a local stdio server for clients without remote support: `uvx --from scrapely-mcp@latest scrapely-mcp-server`. The source is on GitHub at scrapely-app/scrapely-mcp, and the [agents page](https://scrape.ly/agents.md) covers the other ways agents use Scrape.ly. ### A real agent run through MCP On 22 October 2026 we called the [RAG Web Browser](https://scrape.ly/store/querystone/rag-web-browser.md) through the Scrape.ly MCP server, using its "Top pages for a search as Markdown, 3 results" preset. This is the `call-actor` request the agent sends: ```json { "actor": "querystone/rag-web-browser", "input": { "query": "web scraping with python", "maxResults": 3, "scrapingTool": "raw-http", "outputFormats": ["markdown"] }, "waitSecs": 45, "callOptions": { "maxTotalChargeUsd": 0.1 } } ``` The run finished inside the wait: status `SUCCEEDED` after 7.8 seconds, 3 dataset items. The agent then read them with `get-actor-output`: a University of Texas library guide on scraping with Python (8,470 characters of Markdown), the Scrapy project's home page (9,343) and a Medium tutorial (12,761), each with its title, canonical URL and language. The run's log: ```plaintext 20261022T18:29:37.650Z [scrapely.actor] INFO Searching Google in fast search mode; a page switches to browser mode only if fast search mode fails 20261022T18:29:39.910Z [scrapely.actor] INFO Search page start=0 in fast search mode: 10 results 20261022T18:29:42.807Z [scrapely.actor] INFO Clean pages 3 of 3 via fast search mode 20261022T18:29:42.809Z [scrapely.actor] INFO Timing: python 1.0s, imports 0.3s, start 0.2s, search 2.3s, 11 page requests, median 1.6s, max 1.9s, hedged 0 (won 0), total 5.4s ``` It was charged $0.0039: $0.0015 for one Google results page, $0.0010 of compute and $0.0013 of residential proxy traffic. The cap of $0.10 was never close. **Best for:** agents that need real, site-specific data from many sources, with one sign-in and a hard spending limit. ## How to add the Scrape.ly MCP server to Claude Code, Cursor and VS Code **Claude Code.** One command, then sign in with OAuth: ```shell claude mcp add --transport http scrapely https://api.scrape.ly/mcp ``` Run `/mcp` inside Claude Code and choose `scrapely` to sign in. To use a token instead, add `--header "Authorization: Bearer YOUR_TOKEN"`. **Cursor.** Add this to `~/.cursor/mcp.json`, or to `.cursor/mcp.json` in a project. Without `headers`, Cursor offers to sign in with OAuth: ```json { "mcpServers": { "scrapely": { "url": "https://api.scrape.ly/mcp", "headers": { "Authorization": "Bearer YOUR_TOKEN" } } } } ``` **VS Code.** Add this to `.vscode/mcp.json`. It asks for the token once and stores it securely: ```json { "inputs": [ { "type": "promptString", "id": "scrapely-token", "description": "scrape.ly API token", "password": true } ], "servers": { "scrapely": { "type": "http", "url": "https://api.scrape.ly/mcp", "headers": { "Authorization": "Bearer ${input:scrapely-token}" } } } } ``` **Claude Desktop.** Add `https://api.scrape.ly/mcp` as a custom connector under **Customize → Connectors**; Claude signs you in with OAuth. Then ask in plain words: "Use scrapely to run querystone/rag-web-browser for 'web scraping with python', 3 results, and summarize each page in two sentences." Create a token, if you use one, under **Settings → API & Integrations**. Every client and option is in the [MCP docs](https://scrape.ly/docs/mcp.md). ## 2. Firecrawl MCP The [Firecrawl MCP server](https://github.com/firecrawl/firecrawl-mcp-server) brings Firecrawl's API to MCP clients: `firecrawl_scrape` (Markdown or JSON from a URL), `firecrawl_map`, `firecrawl_crawl`, `firecrawl_search`, `firecrawl_interact` for clicking and typing in a real browser, `firecrawl_agent` for multi-source research, monitoring tools and credit-usage checks: 27 tools in the full profile. The hosted endpoint is `https://mcp.firecrawl.dev/v2/mcp`, with an OAuth endpoint for account sign-in and an API-key option. Without a key, a rate-limited keyless tier exposes scrape, search and parse only. It also runs locally with `npx -y firecrawl-mcp` and can point at a self-hosted Firecrawl. Usage bills Firecrawl credits: 1 per page for scrape and crawl, 2 per 10 search results, 4 more per page for JSON. The free plan has 1,000 credits a month; paid plans start at $19 for 5,000 ([Firecrawl pricing explained](https://scrape.ly/blog/firecrawl-pricing.md)). **Best for:** agents that mostly turn known URLs into Markdown. For site-specific data, or a lower price per crawled page, see [Firecrawl alternatives](https://scrape.ly/blog/firecrawl-alternatives.md). ## 3. Bright Data MCP The [Bright Data MCP server](https://github.com/brightdata/brightdata-mcp) has 32 tools: `search_engine` for Google, Bing and Yandex, `scrape_as_markdown` and batch scraping of up to 10 URLs, six tools to find and run 1,200+ pre-built scrapers, and 14 `scraping_browser_*` tools for a remote browser. You connect with your API token in the URL: `https://mcp.brightdata.com/mcp?token=YOUR_API_TOKEN`, or run it locally. Every account gets 5,000 free requests a month. After that it's pay as you go: $1.50 per 1,000 results for search, scraping and extraction, and $8 per GB for browser navigation. **Best for:** agents that need Bright Data's proxy network for protected sites. Note that a token in the URL ends up in logs and history; Scrape.ly's server signs in with OAuth instead. ## 4. Apify MCP The [Apify MCP server](https://docs.apify.com/platform/integrations/mcp) at `https://mcp.apify.com` connects with OAuth or a Bearer token and lets agents search the Apify Store, inspect Actors and run them with `search-actors`, `fetch-actor-details` and `call-actor`. By default it also loads the `apify/rag-web-browser` and `apify/web-fetch` Actors as tools. It allows 30 requests per second per user and has a local stdio version (`@apify/actors-mcp-server`). Runs bill Apify platform usage and Actor fees: the free plan includes $5 a month, and residential proxy costs $7 to $8 per GB. **Best for:** teams already running Actors on Apify. The core tool names on Scrape.ly's server are the same, and Scrape.ly runs most Apify Actors unchanged while charging $2 to $4 per GB for residential proxy. ## 5. Playwright MCP [Playwright MCP](https://github.com/microsoft/playwright-mcp) is Microsoft's browser automation server. It works from Playwright's accessibility tree instead of screenshots, so the model needs no vision: it calls `browser_navigate`, `browser_snapshot`, `browser_click`, `browser_type`, `browser_fill_form`, `browser_tabs` and dozens more. It runs on your machine: ```shell claude mcp add playwright npx @playwright/mcp@latest ``` Pass `--port 8931` to serve it over HTTP. It needs Node.js 18 or newer. Microsoft's own README suggests that coding agents may do better with its Playwright CLI and skills, which avoid loading large tool schemas and accessibility trees into the context. **Best for:** logins, forms and multi-step flows where the agent has to act like a user, and for testing your own web app. We cover it in depth in [Playwright MCP](https://scrape.ly/blog/playwright-mcp.md). For data at volume, pair it with a data server. ## 6. Puppeteer MCP The [reference Puppeteer server](https://github.com/modelcontextprotocol/servers-archived/tree/main/src/puppeteer) gives an agent seven tools: `puppeteer_navigate`, `puppeteer_screenshot`, `puppeteer_click`, `puppeteer_hover`, `puppeteer_fill`, `puppeteer_select` and `puppeteer_evaluate` for running JavaScript in the page, plus console logs and screenshots as resources. It runs with `npx -y @modelcontextprotocol/server-puppeteer` (a visible browser window) or as the `mcp/puppeteer` Docker image (headless Chromium). It's MIT-licensed and now lives in the `modelcontextprotocol/servers-archived` repository. Its README warns that it can reach local files and internal IP addresses, because the browser runs on your machine. **Best for:** quick experiments if you already know Puppeteer. For anything ongoing, use Playwright MCP or a hosted data server. ## Security and cost tips for scraping agents - **Keep tokens out of URLs.** URLs end up in logs and shell history. Prefer OAuth, or a header. - **Give agents their own key.** A scoped key with run and read permissions and a daily charge limit means a confused agent can't empty your balance. - **Cap every run.** Pass `maxTotalChargeUsd` (and `maxItems` for per-result Actors) so a loop of retries stops at a known cost. - **Sandbox local browsers.** Playwright and Puppeteer servers run with your machine's network access. Don't point them at internal tools by accident. - **Collect only public data you need.** Store Actors on Scrape.ly collect publicly available data that any visitor sees. Personal data still falls under privacy laws such as the GDPR and the CCPA. ## FAQ ### What is the best MCP server for web scraping? Scrape.ly. It's hosted, signs in with OAuth, and puts a Store of 568 ready-made scrapers behind three tools, with a dollar cap per run and daily limits per key. Our test run through it returned three search results as Markdown in 7.8 seconds for $0.0039. ### Is there a free web scraping MCP server? Playwright MCP and Puppeteer MCP are free to run on your own machine. Among hosted servers, Scrape.ly includes $5 of usage every month without a card, Firecrawl 1,000 credits a month and Bright Data 5,000 requests a month. ### Which MCP server is best for web search? For search plus page content, Scrape.ly's RAG Web Browser searches Google and returns the top pages as Markdown for $1.50 per 1,000 results pages, and the agent can call it through `call-actor`. Firecrawl and Bright Data also offer search tools. ### Do I need Playwright MCP if I have a scraping MCP server? Only for flows that need clicking, typing or logging in step by step. For extracting data, a hosted server is faster and cheaper per page because the agent gets finished results instead of driving every click. ### Can an AI agent spend money through an MCP server? Yes. Runs started through MCP are normal paid runs. On Scrape.ly you limit that with a scoped API key and a daily charge limit, a maximum charge per run on the key, and `maxTotalChargeUsd` on each call. ### Does the Scrape.ly MCP server work with ChatGPT and Windsurf? Yes. Any client that supports streamable HTTP can use `https://api.scrape.ly/mcp` with OAuth or a Bearer header, including ChatGPT, Codex, Windsurf, Cline and Zed. Clients that only support stdio can use the local server. Scrape.ly Store Give your agent a web browser Search Google and get the top pages as clean Markdown: $1.50 per 1,000 results pages, callable from any MCP client. [Try it free](https://scrape.ly/store/querystone/rag-web-browser.md) [Browse the Store](https://scrape.ly/store.md) About the author [Scrape.ly Team](https://scrape.ly/blog/author/scrapely-team.md) The team behind Scrape.ly Scrape.ly is the web data platform from TeraShift GmbH in Switzerland, powered by Evomi: ready-made scrapers in the Scrape.ly Store, a cloud runtime, an API with JavaScript and Python SDKs, proxies, a hosted anti-bot browser and an MCP server for AI agents. - [Website](https://scrape.ly/index.md) Keep reading ## More in AI agents - [![Pixel-art robot with an orange >\_ on its terminal head, an AI sparkle in a speech bubble above it, pulls a data card out of a web page and drops it into a crate](https://scrape.ly/media/ai-web-scraping.png?w=750)](https://scrape.ly/blog/ai-web-scraping.md) ### [AI Web Scraping: How LLMs and Agents Extract Web Data](https://scrape.ly/blog/ai-web-scraping.md) [AI web scraping uses language models to read pages and fill structured fields, and AI agents to decide what to fetch. Learn how LLM extraction, agents and MCP work, when plain rules still win, and run both with working code and real runs.](https://scrape.ly/blog/ai-web-scraping.md) - [![Pixel-art robot with an orange >\_ on its terminal head carries a cable from a browser window showing the Playwright logo into a server rack labelled MCP](https://scrape.ly/media/playwright-mcp.png?w=750)](https://scrape.ly/blog/playwright-mcp.md) ### [Playwright MCP: Setup for Cursor, Claude Code and VS Code, and When to Use It for Scraping](https://scrape.ly/blog/playwright-mcp.md) [Playwright MCP lets an AI agent drive a real browser through accessibility snapshots. Set it up in Cursor, Claude Code or VS Code in a minute, learn the options and tools that matter, and see where it shines for scraping and where to hand off.](https://scrape.ly/blog/playwright-mcp.md) - [![Pixel-art robot with an orange >\_ on its terminal head, an AI sparkle in a speech bubble above it, reaches out to a standing text file with a folded corner whose dark plate reads LLMS, next to a web page](https://scrape.ly/media/what-is-llms-txt.png?w=750)](https://scrape.ly/blog/what-is-llms-txt.md) ### [What Is llms.txt? Format, Real Examples and How to Create One](https://scrape.ly/blog/what-is-llms-txt.md) [llms.txt is a Markdown file at the root of a website that gives AI agents a short, curated map of its content. The format rules from the v2 proposal, real files from Anthropic, OpenAI, Stripe and Cloudflare, llms-full.txt, Markdown twins, and how to create one in 11 seconds.](https://scrape.ly/blog/what-is-llms-txt.md) ## The internet is full of data. Start owning yours. Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes. The advantage lasts forever. [Get Started Free](https://my.scrape.ly/register) [Chat with Support](mailto:support@scrape.ly?subject=Question%20about%20Scrape.ly\&body=Hi%2C%20I%20have%20a%20question%20about%20Scrape.ly.) No credit card. No setup calls.