Claude MCP Server for Live Web Data: Setup for Claude Desktop and Claude Code

A Claude MCP server is a Model Context Protocol server that you connect to Claude, as a custom connector in Claude Desktop and claude.ai or with claude mcp add in Claude Code, so Claude can call its tools in the middle of a conversation. To give Claude live web data, connect the hosted Scrape.ly MCP server: one URL and an OAuth sign-in let Claude search Google, read articles and run any of the 570+ ready-made scrapers in the Scrape.ly Store, with a dollar cap on every run.

This guide walks through the setup for each Claude app, the four tool calls Claude makes, and a real two-step run from 26 October 2026: Claude found this week's Model Context Protocol news on Google and read three of the articles, in under ten seconds of run time, for less than a cent.

What a Claude MCP server is

MCP is the open standard Anthropic created for connecting AI apps to tools and data. A server publishes a set of tools, each with a name, a description and an input schema. When you connect it, Claude sees those tools and decides when to call them; you approve the calls, and the results come back into the conversation.

Some MCP servers act on your own accounts (GitHub, Notion, a database). A web data server does something different: it fetches public information Claude couldn't otherwise get in a structured form. With Scrape.ly that's a whole catalog behind one server:

Prices are pay per result with platform usage included, as listed on each Store page on 26 October 2026. Claude finds the right Actor itself with search-actors, so the list isn't limited to these five.

Add the Scrape.ly MCP server to Claude Desktop and claude.ai

Custom connectors work in Claude Desktop, claude.ai and Cowork, and they're configured once in your Claude account. Create a free Scrape.ly account first (the free plan includes $5 of usage every month, no card needed). Then, on a Pro or Max plan:

1. Open Customize → Connectors.
2. Click + Add, then Add custom connector.
3. Name it Scrape.ly and enter the URL https://mcp.scrape.ly. Click Continue.
4. Review the authentication settings Claude detected (OAuth) and click Continue. If Claude asks how to identify itself, Register automatically uses the dynamic client registration the Scrape.ly server supports.
5. Click Add, sign in to Scrape.ly in the browser window that opens, and approve access.

In a chat, turn the connector on with the + button at the bottom left, then Connectors. Anthropic's help center notes that Free plan users are limited to one custom connector. On Team and Enterprise plans, an Owner adds the connector once under Organization settings → Connectors, and each member then clicks Connect to sign in with their own Scrape.ly account.

One detail matters for any remote connector: Claude connects to the server from Anthropic's cloud, not from your computer. That's no problem for Scrape.ly's server, which is public at https://mcp.scrape.ly, and it means the scraping never runs on your laptop or your network.

Add it to Claude Code

In Claude Code it's one command:

claude mcp add --transport http scrapely https://mcp.scrape.ly

Run /mcp inside Claude Code, choose scrapely and sign in with OAuth. Claude Code stores the tokens and refreshes them for you. To use an API token instead, add --header "Authorization: Bearer YOUR_TOKEN" with a token from Settings → API & Integrations.

Scope decides where the server is available. The default is local scope: only the project you added it in. Add --scope user to have it in every project on your machine, or --scope project to write it to a .mcp.json file that you commit so your team gets the same server. Check the connection with:

claude mcp list

A healthy server shows ✔ Connected; ! Needs authentication means you haven't finished the /mcp sign-in.

Give Claude only the tools it needs

By default the server exposes its general tools and Claude picks Actors from the Store. To pin specific scrapers as their own tools, list them in the URL (up to 50):

claude mcp add --transport http scrapely-research "https://mcp.scrape.ly?actors=querystone/google-search-scraper,querystone/article-extractor"

The same URL works as a custom connector in Claude Desktop. A short tool list keeps Claude's choices obvious and its context small.

What Claude does with it: the four tool calls

Every web data request goes through the same four tools:

1. search-actors finds candidate Actors in the Store for the task ("google search", "zillow").
2. fetch-actor-details reads the chosen Actor's input schema, README and price, so Claude knows which fields to send.
3. call-actor starts the run with a JSON input and waits up to 45 seconds. Most single-page jobs finish inside that; for longer runs Claude follows up with get-actor-run.
4. get-actor-output reads the results page by page, with only the fields Claude asks for.

The rows never arrive inside the call-actor reply. That matters in Claude Code, which warns when one tool result passes 10,000 tokens and cuts it at 25,000 by default (you can raise the limit with MAX_MCP_OUTPUT_TOKENS). Because get-actor-output pages through datasets, about 50 KB at a time, with long text values trimmed and only the fields Claude names, a 5,000-row dataset never hits that ceiling.

A real run: this week's MCP news, searched and read

The prompt: "Use scrapely to find news about the Model Context Protocol from the past week, read the three most useful articles and tell me what changed."

Step 1: search. Claude picked the Google Search Scraper and sent this call-actor request, with Google's past-week filter and a cap on the charge:

{
  "actor": "querystone/google-search-scraper",
  "input": {
    "queries": "model context protocol news",
    "countryCode": "us",
    "languageCode": "en",
    "maxPagesPerQuery": 1,
    "resultsPerPage": 10,
    "quickDateRange": "w"
  },
  "waitSecs": 45,
  "callOptions": { "maxTotalChargeUsd": 0.05 }
}

The run took 4.2 seconds: "1 pages, 10 organic results, 0 failed". Five of the ten results, as get-actor-output returned them:

Step 2: read. Claude chose three articles (the SDK release, the security course and the sources roundup) and passed their URLs to the Article Extractor:

{
  "actor": "querystone/article-extractor",
  "input": {
    "articleUrls": [
      "https://www.infoq.com/news/2026/10/MCP-CSharp-SDK-Stateless/",
      "https://www.freecodecamp.org/news/learn-model-context-protocol-mcp-security/",
      "https://daily.dev/blog/best-sources-ai-agents-mcp-news/"
    ],
    "includeMarkdown": true
  },
  "waitSecs": 45,
  "callOptions": { "maxTotalChargeUsd": 0.05 }
}

It finished in 5.5 seconds. The log:

2026-10-26T14:26:42.322Z [scrapely.actor] INFO  Extracting articles from 0 start URL(s) and 3 article URL(s) via datacenter proxy.
2026-10-26T14:26:44.224Z [scrapely.actor] INFO  3 article(s) extracted. Downloaded 0.7 MB.

What came back, one row per article:

Each row also carried the article's description, language, canonical URL and the full body as Markdown, all three fetched over plain HTTP without a browser. That's what Claude answered from. The InfoQ piece reports that version 2.0 of the official MCP C# SDK implements the 2026-07-28 revision of the specification, makes Streamable HTTP servers stateless by default and adds Multi Round-Trip Requests for interactive tools. freeCodeCamp announced a two-hour MCP security course on its YouTube channel, and the daily.dev roundup recommends starting with the MCP specification changelog.

Cost. The search was 1 page × $0.0033, the reading 3 articles × $0.0009 plus $0.003 for the run: $0.009 for both steps at list price, platform usage included. Our account was billed $0.0072 after its plan's Store discount.

More things to ask Claude

Once the connector is on, plain requests work. Naming the Actor and a size makes Claude faster and cheaper:

• "Use scrapely to get the 10 best-rated coffee shops in Portland, OR from Google Maps with their hours." Our Google Maps MCP guide shows that run.
• "What did NASA post on YouTube this month, and which video has the most views per day?" See YouTube MCP.
• "Find five homes for sale in Boise, ID and compare each asking price with its Zestimate." See Zillow MCP.
• "Search Google in Germany for 'wärmepumpe kosten' and list the People Also Ask questions."
• "Keep every run under $0.10."

Claude turns that last line into callOptions.maxTotalChargeUsd on each call-actor.

Spending control

An assistant that can start paid runs needs limits, and there are three:

• A dollar cap per run. maxTotalChargeUsd caps everything a run can cost; pay-per-result Actors stop cleanly at the cap and keep what they saved.
• A scoped API key. For Claude Code jobs that run unattended, create a key under Settings → API keys with only actors:run and runs:read and a daily charge limit, and pass it with --header.
• A prepaid balance. Usage draws from your balance; nothing is billed after the fact.

What to watch out for

• Connect servers you trust. Anthropic's guidance for custom connectors is to connect only servers from organizations you trust, watch for prompt injection in tool results, and click "Allow always" only for tools you're happy to run unsupervised. The Scrape.ly server's tools search the Store, run Actors and read your own runs and storage, so they act on your Scrape.ly account and nothing else.
• Long runs. A crawl of hundreds of pages takes longer than the 45-second wait. That's normal: Claude polls get-actor-run until it finishes, or you ask it to check back later.
• Ask for fields, not everything. "Give me title, date and URL" keeps results small and well under Claude Code's output limit.
• Public data, used responsibly. Store Actors collect publicly available data that any visitor sees. Reviews, comments and profiles are often about people, so collect only what the task needs; our guide to web scraping and the law covers the basics. This isn't legal advice.

For other clients and options, including VS Code, Cursor and a local stdio server, see the MCP docs and the Scrape.ly MCP page. To compare servers, read the best MCP servers for web scraping and, for search specifically, the best web search MCP servers.

FAQ

What are Claude MCP servers?

They're MCP servers connected to Claude, either as custom connectors in Claude Desktop and claude.ai or through claude mcp add in Claude Code. Each server gives Claude a set of tools; a web data server like Scrape.ly's lets Claude search Google, read pages and run ready-made scrapers.

Is the Claude MCP server free?

Connecting a server costs nothing on Claude's side, and Free plan users can add one custom connector. On Scrape.ly the server is free too: you pay only for the runs Claude starts, and the free plan includes $5 of usage every month. Our search-and-read run cost $0.009 at list price.

What is the Claude MCP server URL for Scrape.ly?

https://mcp.scrape.ly. Use it as the custom connector URL in Claude Desktop and claude.ai, or with claude mcp add --transport http scrapely https://mcp.scrape.ly in Claude Code. Add ?actors=owner/name,… to expose only the scrapers you want.

How do I add an MCP server to Claude Code?

Run claude mcp add --transport http <name> <url>, then /mcp inside Claude Code to sign in. The server lands in local scope by default; add --scope user for all your projects or --scope project to share it through .mcp.json.

Can Claude browse the web with MCP?

Yes. With a web data server connected, Claude can search Google, open and read articles, and pull structured data from sites like Google Maps, YouTube or Zillow, all as tool calls you can see and approve in the chat.