--- title: "Claude MCP Server for Live Web Data | Scrape.ly" description: "Add a Claude MCP server for live web data to Claude Desktop or Claude Code: setup steps, the tool calls, a real search-and-read run and its cost." url: https://scrape.ly/blog/claude-mcp-web-data markdown: https://scrape.ly/blog/claude-mcp-web-data.md --- [AI agents](https://scrape.ly/blog/category/ai-agents.md) # Claude MCP Server for Live Web Data: Setup for Claude Desktop and Claude Code Connect a Claude MCP server that gives Claude live web data: Google results, full articles, Maps places, YouTube videos and more. Step-by-step setup for Claude Desktop, claude.ai and Claude Code, the exact tool calls, a real search-and-read run and what it cost. [Scrape.ly Team The team behind Scrape.ly](https://scrape.ly/blog/author/scrapely-team.md) - Published Oct 26, 2026 - Reading time 9 min read ![Pixel-art robot with an orange >\_ on its terminal head carries a cable from a web page into a server rack labelled MCP, with the orange Claude spark in a speech bubble above the rack](https://scrape.ly/media/claude-mcp-web-data.png?w=1080) A Claude MCP server is a Model Context Protocol server that you connect to Claude, as a custom connector in Claude Desktop and claude.ai or with `claude mcp add` in Claude Code, so Claude can call its tools in the middle of a conversation. To give Claude live web data, connect the hosted Scrape.ly MCP server: one URL and an OAuth sign-in let Claude search Google, read articles and run any of the 570+ ready-made scrapers in the Scrape.ly Store, with a dollar cap on every run. This guide walks through the setup for each Claude app, the four tool calls Claude makes, and a real two-step run from 26 October 2026: Claude found this week's Model Context Protocol news on Google and read three of the articles, in under ten seconds of run time, for less than a cent. ## What a Claude MCP server is MCP is the open standard Anthropic created for connecting AI apps to tools and data. A server publishes a set of tools, each with a name, a description and an input schema. When you connect it, Claude sees those tools and decides when to call them; you approve the calls, and the results come back into the conversation. Some MCP servers act on your own accounts (GitHub, Notion, a database). A web data server does something different: it fetches public information Claude couldn't otherwise get in a structured form. With Scrape.ly that's a whole catalog behind one server: | You ask Claude for | Actor it runs | Price | | -------------------------------------------------------------- | ------------------------------------------------------------------------------------ | ----------------------------------------- | | Google results for a query, with positions and People Also Ask | [Google Search Scraper](https://scrape.ly/store/querystone/google-search-scraper.md) | $3.30 per 1,000 pages | | The full text of articles, as Markdown | [Article Extractor](https://scrape.ly/store/querystone/article-extractor.md) | $0.90 per 1,000 articles + $0.003 per run | | Places with ratings, hours and phone numbers | Google Maps Scraper Lite | $3.81 per 1,000 places | | A channel's latest videos with views and likes | YouTube Scraper | $3.95 per 1,000 videos | | Homes for sale with price and Zestimate | Zillow Search Scraper | $1.51 per 1,000 listings | Prices are pay per result with platform usage included, as listed on each Store page on 26 October 2026. Claude finds the right Actor itself with `search-actors`, so the list isn't limited to these five. ## Add the Scrape.ly MCP server to Claude Desktop and claude.ai Custom connectors work in Claude Desktop, claude.ai and Cowork, and they're configured once in your Claude account. Create a free Scrape.ly account first (the free plan includes $5 of usage every month, no card needed). Then, on a Pro or Max plan: 1. Open **Customize → Connectors**. 2. Click **+ Add**, then **Add custom connector**. 3. Name it `Scrape.ly` and enter the URL `https://mcp.scrape.ly`. Click **Continue**. 4. Review the authentication settings Claude detected (OAuth) and click **Continue**. If Claude asks how to identify itself, **Register automatically** uses the dynamic client registration the Scrape.ly server supports. 5. Click **Add**, sign in to Scrape.ly in the browser window that opens, and approve access. In a chat, turn the connector on with the **+** button at the bottom left, then **Connectors**. Anthropic's help center notes that Free plan users are limited to one custom connector. On Team and Enterprise plans, an Owner adds the connector once under **Organization settings → Connectors**, and each member then clicks **Connect** to sign in with their own Scrape.ly account. One detail matters for any remote connector: Claude connects to the server from Anthropic's cloud, not from your computer. That's no problem for Scrape.ly's server, which is public at `https://mcp.scrape.ly`, and it means the scraping never runs on your laptop or your network. ## Add it to Claude Code In Claude Code it's one command: ```shell claude mcp add --transport http scrapely https://mcp.scrape.ly ``` Run `/mcp` inside Claude Code, choose `scrapely` and sign in with OAuth. Claude Code stores the tokens and refreshes them for you. To use an API token instead, add `--header "Authorization: Bearer YOUR_TOKEN"` with a token from **Settings → API & Integrations**. Scope decides where the server is available. The default is local scope: only the project you added it in. Add `--scope user` to have it in every project on your machine, or `--scope project` to write it to a `.mcp.json` file that you commit so your team gets the same server. Check the connection with: ```shell claude mcp list ``` A healthy server shows `✔ Connected`; `! Needs authentication` means you haven't finished the `/mcp` sign-in. ### Give Claude only the tools it needs By default the server exposes its general tools and Claude picks Actors from the Store. To pin specific scrapers as their own tools, list them in the URL (up to 50): ```shell claude mcp add --transport http scrapely-research "https://mcp.scrape.ly?actors=querystone/google-search-scraper,querystone/article-extractor" ``` The same URL works as a custom connector in Claude Desktop. A short tool list keeps Claude's choices obvious and its context small. ## What Claude does with it: the four tool calls Every web data request goes through the same four tools: 1. **`search-actors`** finds candidate Actors in the Store for the task ("google search", "zillow"). 2. **`fetch-actor-details`** reads the chosen Actor's input schema, README and price, so Claude knows which fields to send. 3. **`call-actor`** starts the run with a JSON input and waits up to 45 seconds. Most single-page jobs finish inside that; for longer runs Claude follows up with `get-actor-run`. 4. **`get-actor-output`** reads the results page by page, with only the fields Claude asks for. The rows never arrive inside the `call-actor` reply. That matters in Claude Code, which warns when one tool result passes 10,000 tokens and cuts it at 25,000 by default (you can raise the limit with `MAX_MCP_OUTPUT_TOKENS`). Because `get-actor-output` pages through datasets, about 50 KB at a time, with long text values trimmed and only the fields Claude names, a 5,000-row dataset never hits that ceiling. ## A real run: this week's MCP news, searched and read The prompt: "Use scrapely to find news about the Model Context Protocol from the past week, read the three most useful articles and tell me what changed." **Step 1: search.** Claude picked the Google Search Scraper and sent this `call-actor` request, with Google's past-week filter and a cap on the charge: ```json { "actor": "querystone/google-search-scraper", "input": { "queries": "model context protocol news", "countryCode": "us", "languageCode": "en", "maxPagesPerQuery": 1, "resultsPerPage": 10, "quickDateRange": "w" }, "waitSecs": 45, "callOptions": { "maxTotalChargeUsd": 0.05 } } ``` The run took 4.2 seconds: "1 pages, 10 organic results, 0 failed". Five of the ten results, as `get-actor-output` returned them: | Position | Site | Title | Date | | -------- | ----------------- | -------------------------------------------------------- | ---------- | | 1 | freecodecamp.org | Learn Model Context Protocol (MCP) Security | 2026-10-22 | | 3 | globenewswire.com | Notified Opens Model Context Protocol (MCP) Beta Program | 2026-10-21 | | 4 | bitsight.com | Model Context Protocol (MCP): A Guide to AI Integration | 2026-10-24 | | 6 | infoq.com | MCP C# SDK 2.0 Makes HTTP Stateless by Default and ... | 2026-10-26 | | 8 | daily.dev | The best sources for AI agents and MCP news in 2026 | 2026-10-21 | **Step 2: read.** Claude chose three articles (the SDK release, the security course and the sources roundup) and passed their URLs to the Article Extractor: ```json { "actor": "querystone/article-extractor", "input": { "articleUrls": [ "https://www.infoq.com/news/2026/10/MCP-CSharp-SDK-Stateless/", "https://www.freecodecamp.org/news/learn-model-context-protocol-mcp-security/", "https://daily.dev/blog/best-sources-ai-agents-mcp-news/" ], "includeMarkdown": true }, "waitSecs": 45, "callOptions": { "maxTotalChargeUsd": 0.05 } } ``` It finished in 5.5 seconds. The log: ```plaintext 20261026T14:26:42.322Z [scrapely.actor] INFO Extracting articles from 0 start URL(s) and 3 article URL(s) via datacenter proxy. 20261026T14:26:44.224Z [scrapely.actor] INFO 3 article(s) extracted. Downloaded 0.7 MB. ``` What came back, one row per article: | Site | Title | Published (UTC) | Words | Read time | | ---------------- | --------------------------------------------------------------------------------- | --------------- | ----- | --------- | | InfoQ | MCP C# SDK 2.0 Makes HTTP Stateless by Default and Adds Multi Round-Trip Requests | 2026-10-26 | 622 | 3 min | | freeCodeCamp.org | Learn Model Context Protocol (MCP) Security | 2026-10-23 | 216 | 1 min | | daily.dev | The best sources for AI agents and MCP news in 2026 | 2026-10-21 | 1,604 | 7 min | Each row also carried the article's description, language, canonical URL and the full body as Markdown, all three fetched over plain HTTP without a browser. That's what Claude answered from. The InfoQ piece reports that version 2.0 of the official MCP C# SDK implements the 2026-07-28 revision of the specification, makes Streamable HTTP servers stateless by default and adds Multi Round-Trip Requests for interactive tools. freeCodeCamp announced a two-hour MCP security course on its YouTube channel, and the daily.dev roundup recommends starting with the MCP specification changelog. **Cost.** The search was 1 page × $0.0033, the reading 3 articles × $0.0009 plus $0.003 for the run: $0.009 for both steps at list price, platform usage included. Our account was billed $0.0072 after its plan's Store discount. Scrape.ly Store Let Claude search Google Google's real results page as JSON, with positions, People Also Ask and related searches. $3.30 per 1,000 pages, from Claude Desktop, claude.ai or Claude Code. [Try it free](https://scrape.ly/store/querystone/google-search-scraper.md) [Browse the Store](https://scrape.ly/store.md) ## More things to ask Claude Once the connector is on, plain requests work. Naming the Actor and a size makes Claude faster and cheaper: - "Use scrapely to get the 10 best-rated coffee shops in Portland, OR from Google Maps with their hours." Our [Google Maps MCP guide](https://scrape.ly/blog/google-maps-mcp.md) shows that run. - "What did NASA post on YouTube this month, and which video has the most views per day?" See [YouTube MCP](https://scrape.ly/blog/youtube-mcp.md). - "Find five homes for sale in Boise, ID and compare each asking price with its Zestimate." See [Zillow MCP](https://scrape.ly/blog/zillow-mcp.md). - "Search Google in Germany for 'wärmepumpe kosten' and list the People Also Ask questions." - "Keep every run under $0.10." Claude turns that last line into `callOptions.maxTotalChargeUsd` on each `call-actor`. ## Spending control An assistant that can start paid runs needs limits, and there are three: - **A dollar cap per run.** `maxTotalChargeUsd` caps everything a run can cost; pay-per-result Actors stop cleanly at the cap and keep what they saved. - **A scoped API key.** For Claude Code jobs that run unattended, create a key under **Settings → API keys** with only `actors:run` and `runs:read` and a daily charge limit, and pass it with `--header`. - **A prepaid balance.** Usage draws from your balance; nothing is billed after the fact. ## What to watch out for - **Connect servers you trust.** Anthropic's guidance for custom connectors is to connect only servers from organizations you trust, watch for prompt injection in tool results, and click "Allow always" only for tools you're happy to run unsupervised. The Scrape.ly server's tools search the Store, run Actors and read your own runs and storage, so they act on your Scrape.ly account and nothing else. - **Long runs.** A crawl of hundreds of pages takes longer than the 45-second wait. That's normal: Claude polls `get-actor-run` until it finishes, or you ask it to check back later. - **Ask for fields, not everything.** "Give me title, date and URL" keeps results small and well under Claude Code's output limit. - **Public data, used responsibly.** Store Actors collect publicly available data that any visitor sees. Reviews, comments and profiles are often about people, so collect only what the task needs; our guide to [web scraping and the law](https://scrape.ly/blog/is-web-scraping-legal.md) covers the basics. This isn't legal advice. For other clients and options, including VS Code, Cursor and a local stdio server, see the [MCP docs](https://scrape.ly/docs/mcp.md) and the [Scrape.ly MCP](https://scrape.ly/mcp.md) page. To compare servers, read the [best MCP servers for web scraping](https://scrape.ly/blog/best-mcp-servers-for-web-scraping.md) and, for search specifically, the [best web search MCP servers](https://scrape.ly/blog/web-search-mcp-server.md). Scrape.ly Store Let Claude read the article, not just the link Clean article text and Markdown with title, author, publish date and language, from any news site, blog or feed. $0.90 per 1,000 articles plus $0.003 per run. [Try it free](https://scrape.ly/store/querystone/article-extractor.md) [Browse the Store](https://scrape.ly/store.md) ## FAQ ### What are Claude MCP servers? They're MCP servers connected to Claude, either as custom connectors in Claude Desktop and claude.ai or through `claude mcp add` in Claude Code. Each server gives Claude a set of tools; a web data server like Scrape.ly's lets Claude search Google, read pages and run ready-made scrapers. ### Is the Claude MCP server free? Connecting a server costs nothing on Claude's side, and Free plan users can add one custom connector. On Scrape.ly the server is free too: you pay only for the runs Claude starts, and the free plan includes $5 of usage every month. Our search-and-read run cost $0.009 at list price. ### What is the Claude MCP server URL for Scrape.ly? `https://mcp.scrape.ly`. Use it as the custom connector URL in Claude Desktop and claude.ai, or with `claude mcp add --transport http scrapely https://mcp.scrape.ly` in Claude Code. Add `?actors=owner/name,…` to expose only the scrapers you want. ### How do I add an MCP server to Claude Code? Run `claude mcp add --transport http `, then `/mcp` inside Claude Code to sign in. The server lands in local scope by default; add `--scope user` for all your projects or `--scope project` to share it through `.mcp.json`. ### Can Claude browse the web with MCP? Yes. With a web data server connected, Claude can search Google, open and read articles, and pull structured data from sites like Google Maps, YouTube or Zillow, all as tool calls you can see and approve in the chat. About the author [Scrape.ly Team](https://scrape.ly/blog/author/scrapely-team.md) The team behind Scrape.ly Scrape.ly is the web data platform from TeraShift GmbH in Switzerland, powered by Evomi: ready-made scrapers in the Scrape.ly Store, a cloud runtime, an API with JavaScript and Python SDKs, proxies, a hosted anti-bot browser and an MCP server for AI agents. - [Website](https://scrape.ly/index.md) Keep reading ## More in AI agents - [![Pixel-art robot with an orange >\_ on its terminal head carries a cable from a stack of cardboard boxes marked with the Amazon logo into a server rack labelled MCP](https://scrape.ly/media/amazon-mcp.png?w=750)](https://scrape.ly/blog/amazon-mcp.md) ### [Amazon MCP: Product, Price and Review Data for AI Agents](https://scrape.ly/blog/amazon-mcp.md) [Amazon's own MCP servers are for ads and AWS. To give Claude, Cursor or ChatGPT live Amazon prices, ratings and reviews, connect Scrape.ly's hosted MCP server and its Amazon scrapers. Setup, tool calls, real keyboard comparisons and exact costs.](https://scrape.ly/blog/amazon-mcp.md) - [![Pixel-art robot with an orange >\_ on its terminal head carries a cable from a feed of posts headed by the white X logo into a server rack labelled MCP](https://scrape.ly/media/twitter-mcp.png?w=750)](https://scrape.ly/blog/twitter-mcp.md) ### [Twitter (X) MCP: Give Your AI Agent Public Posts Without an X Developer App](https://scrape.ly/blog/twitter-mcp.md) [A Twitter MCP server lets Claude, Cursor or ChatGPT read public X posts and their engagement counts as tool calls. Connect Scrape.ly's hosted server in a minute, with no X developer app or API keys, and see the exact tool calls, real output from @NASA and @SpaceX, and what it cost.](https://scrape.ly/blog/twitter-mcp.md) - [![Pixel-art robot with an orange >\_ on its terminal head carries a cable from a monitor showing the red YouTube logo above a video thumbnail and its title into a server rack labelled MCP](https://scrape.ly/media/youtube-mcp.png?w=750)](https://scrape.ly/blog/youtube-mcp.md) ### [YouTube MCP: Video, Channel and Comment Data for Claude, ChatGPT and Cursor](https://scrape.ly/blog/youtube-mcp.md) [A YouTube MCP server lets your AI agent read public videos, channels, captions and comments right in the chat. Connect Scrape.ly's hosted server to Claude, ChatGPT or Cursor without a YouTube Data API key or quota, and see the exact tool calls, real output and costs.](https://scrape.ly/blog/youtube-mcp.md) ## The internet is full of data. Start owning yours. Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes. [Get Started Free](https://my.scrape.ly/register) [Chat with Support](mailto:hello@scrape.ly?subject=Question%20about%20Scrape.ly\&body=Hi%2C%20I%20have%20a%20question%20about%20Scrape.ly.) No credit card. No setup calls.