--- title: "ScrapeGraphAI Tutorial: Setup, Code, Costs | Scrape.ly" description: "ScrapeGraphAI tutorial tested on v2.3.1: install, SmartScraperGraph, schemas, local and hosted LLMs, real costs, and a hosted route with no LLM tokens." url: https://scrape.ly/blog/scrapegraphai-guide markdown: https://scrape.ly/blog/scrapegraphai-guide.md --- [AI agents](https://scrape.ly/blog/category/ai-agents.md) # ScrapeGraphAI Tutorial: How It Works, a Tested Setup and a Hosted Alternative ScrapeGraphAI turns a prompt and a URL into JSON by sending the page to an LLM. We installed version 2.3.1, traced a SmartScraperGraph run node by node, priced its hosted API, and show the Scrape.ly route that returns structured data without paying for LLM tokens on every page. [Scrape.ly Team The team behind Scrape.ly](https://scrape.ly/blog/author/scrapely-team.md) - Published Oct 26, 2026 - Reading time 9 min read ![Pixel-art robot with an orange >\_ on its terminal head pulls a card with curly braces out of a monitor showing the purple ScrapeGraphAI spider logo and a graph of nodes that runs from a small web page to curly braces, and drops it into a crate](https://scrape.ly/media/scrapegraphai-guide.png?w=1080) ScrapeGraphAI is an open-source Python library that extracts data from web pages by sending their content to a large language model with your prompt. You give ScrapeGraphAI a URL and a sentence such as "list every book with its price", and a small graph of steps fetches the page, cleans it, and asks the model for JSON. This tutorial covers what you need to judge it for real work: installation, the `SmartScraperGraph` pipeline, schemas, local and hosted models, the hosted ScrapeGraphAI API and its credit prices, and where an LLM per page stops making sense. We installed version 2.3.1 on 26 October 2026 and ran it against the practice site books.toscrape.com; we show exactly what happened, including the step that needs a model. The last part covers the Scrape.ly route, which returns structured data from the same kinds of pages with no LLM tokens at all. ## ScrapeGraphAI at a glance | | | | -------------- | -------------------------------------------------------------------------------------------- | | What it is | A Python library of scraping "graphs": pipelines of nodes that fetch, parse and ask an LLM | | Version | 2.3.1, released 6 October 2026 ([PyPI](https://pypi.org/project/scrapegraphai/)) | | License | MIT ([GitHub](https://github.com/ScrapeGraphAI/Scrapegraph-ai)) | | Python | 3.12 or newer | | LLMs | OpenAI, Groq, Azure, Gemini, MiniMax and others through APIs, or local models through Ollama | | Main pipeline | `SmartScraperGraph`: one page, one prompt, JSON out | | Browser | Playwright (`playwright install` after `pip install`) | | Telemetry | On by default; opt out with `SCRAPEGRAPHAI_TELEMETRY_ENABLED=false` | | Hosted version | ScrapeGraphAI API with the `scrapegraph-py` SDK, billed in credits | ## How to install ScrapeGraphAI The README installs the package and then the browser it uses to fetch pages: ```shell pip install scrapegraphai playwright install ``` ScrapeGraphAI is built on LangChain, so the install is large. In a clean environment, `uv` resolved 98 packages for version 2.3.1, including the LangChain integrations for OpenAI, AWS, Mistral and Ollama, Playwright and `undetected-playwright`. The first `import` of `SmartScraperGraph` took 21.6 seconds on our machine. To try it without touching your global Python, this is the command we used: ```shell SCRAPEGRAPHAI_TELEMETRY_ENABLED=false uv run --python 3.12 --with scrapegraphai==2.3.1 python first_graph.py ``` ## Your first graph: SmartScraperGraph `SmartScraperGraph` takes a prompt, a source URL and a config whose `llm` block picks the model. This is the README's shape, pointed at books.toscrape.com and a local Llama model through Ollama: ```python import json from scrapegraphai.graphs import SmartScraperGraph graph_config = { "llm": {"model": "ollama/llama3.2", "model_tokens": 8192, "format": "json"}, "headless": True, "verbose": True, } graph = SmartScraperGraph( prompt="List every book on the page with its title and price", source="https://books.toscrape.com/", config=graph_config, ) print(json.dumps(graph.run(), indent=2)) ``` With `verbose` on, the run prints each node as it executes. Here is what our machine printed, without an Ollama server running: ```plaintext Executing Fetch Node (Fetching HTML from: https://books.toscrape.com/) --- Executing ParseNode Node Executing GenerateAnswer Node Error during chain execution: [Errno 61] Connection refused ``` That trace is the whole design in four lines. The **Fetch** node loaded the page in a browser, the **Parse** node turned the HTML into text chunks, and the **GenerateAnswer** node tried to send those chunks to the model, which wasn't there. Every extraction in ScrapeGraphAI ends in that LLM call. There is no selector-based path to fall back on, so you need either a local model (install [Ollama](https://ollama.com/) and `ollama pull llama3.2`) or an API key before the library returns anything. To use a hosted model instead, change only the `llm` block, as the README shows: ```python graph_config = { "llm": {"api_key": "YOUR_OPENAI_API_KEY", "model": "openai/gpt-4o-mini"}, "verbose": True, "headless": False, } ``` We didn't run the OpenAI variant: it needs a paid key, and the output would depend on the model rather than on the library. ## Get typed output with a schema Free-form JSON from an LLM changes shape between runs: one call says `price`, the next says `book_price`. In 2.3.1, `SmartScraperGraph` takes an optional `schema` argument (a Pydantic model class), which tells the model the exact shape to return: ```python from typing import List from pydantic import BaseModel from scrapegraphai.graphs import SmartScraperGraph class Book(BaseModel): title: str price: str class Books(BaseModel): books: List[Book] graph = SmartScraperGraph( prompt="List every book on the page with its title and price", source="https://books.toscrape.com/", config=graph_config, schema=Books, ) ``` A schema narrows the output; it doesn't make it correct. The model can still skip a row or misread a price, so validate the result before you store it, and spot-check it against the page. ## The other ScrapeGraphAI pipelines Version 2.3.1 exports 24 graph classes. The README highlights six: | Pipeline | What it does | | ------------------------- | ----------------------------------------------------------- | | `SmartScraperGraph` | One page, one prompt | | `SmartScraperMultiGraph` | One prompt over a list of pages, with LLM calls in parallel | | `SearchGraph` | Extracts from the top results of a search engine | | `ScriptCreatorGraph` | Writes a Python scraping script for a page | | `ScriptCreatorMultiGraph` | Writes a script that covers several pages | | `SpeechGraph` | Extracts from a page and generates an audio file | The rest cover local files (`CSVScraperGraph`, `JSONScraperGraph`, `XMLScraperGraph`, `DocumentScraperGraph`), screenshots, depth crawling (`DepthSearchGraph`) and "lite" variants. `ScriptCreatorGraph` is the most interesting for production: it uses the LLM once to write a scraper, and the script then runs without a model. That's the same idea behind every approach below that doesn't pay per page. ## The hosted ScrapeGraphAI API ScrapeGraphAI also sells a managed API, used through the `scrapegraph-py` and `scrapegraph-js` SDKs. The README describes the split: with the open-source library you bring your own LLM and handle browsers, proxies and scaling; with the API those are managed and you pay in credits. The SDK call is short: ```python from scrapegraph_py import ScrapeGraphAI sgai = ScrapeGraphAI() # reads SGAI_API_KEY res = sgai.extract("Extract product names and prices", url="https://example.com") print(res.data.json_data if res.status == "success" else res.error) ``` The [pricing page](https://scrapegraphai.com/pricing) (checked 26 October 2026) lists a free plan with 500 credits, one time, and paid plans at $20 a month for 10,000 credits, $100 for 100,000 and $500 for 750,000. An Extract call costs 5 credits, and turning on stealth (anti-bot bypass) adds 5 more ([Extract docs](https://docs.scrapegraphai.com/services/extract)). On the $20 plan, a credit costs $0.002, so: - 1,000 pages through Extract = 5,000 credits = **$10**. - 1,000 pages through Extract with stealth = 10,000 credits = **$20**, the whole monthly allowance. ## What to watch out for with an LLM on every page ScrapeGraphAI is quick to prototype with, because you write a sentence instead of selectors. The bill for that convenience arrives in production: - **Tokens on every page.** The open-source library's cost model, in its own README, is "LLM tokens + your own infra". Every page you extract is another model call, forever, even when the layout never changes. - **Output that drifts.** Two runs on the same page can return different field names, a missing row or a rounded price. A schema and validation reduce this; they don't remove it. - **Slow by design.** A model call per page is slower than parsing HTML, and rate limits on your LLM provider become rate limits on your scraper. - **Browsers, proxies and blocking.** In the open-source library, the README lists proxies, anti-bot handling, scaling and maintenance as your responsibility. - **Telemetry.** The library collects anonymous usage metrics unless you set `SCRAPEGRAPHAI_TELEMETRY_ENABLED=false`. - **Rules and privacy.** Scrape public pages, respect rate limits and site terms, and treat personal data under the GDPR and CCPA. Most pages you'll want to scrape at scale don't need a model to understand them. Product, recipe, event and article pages usually publish their data as schema.org markup for search engines, and the rest have a stable layout that a page function reads for a fraction of a cent. ## The Scrape.ly route: structured data with no LLM tokens Scrape.ly runs ready-made scrapers, called Actors, in the cloud. You call them from the console, the REST API, Python or Node.js clients, or an AI agent over MCP, and every Actor has a fixed price per result. Three of them replace a ScrapeGraphAI pipeline, depending on the page. **1. Pages that publish structured data.** The [Structured Data Extractor](https://scrape.ly/store/querystone/structured-data-extractor.md) reads JSON-LD, Microdata, RDFa, Open Graph and meta tags, and normalises the page's main entity into flat columns: price, currency, availability, rating, author, dates, address. No model is involved, so the same page returns the same row every time. On 23 October 2026 we ran its preset "Recipe, product and movie pages, 3 URLs": | Page | Type | Name | Price | Rating | Reviews | | ------------------ | ------- | ------------------------------- | ------- | ------ | ------- | | allrecipes.com | Recipe | Spinach and Feta Turkey Burgers | — | 4.6 | 862 | | apple.com | Product | iPhone 16 | 799 USD | — | — | | rottentomatoes.com | Movie | The Shawshank Redemption | — | 89 | 147 | It finished in 9.1 seconds. At today's list price, three pages cost 3 × $0.00111 = $0.0033, platform usage included: $1.11 per 1,000 pages, against $10 for 1,000 Extract calls on the hosted ScrapeGraphAI plan above. **2. Pages with a stable layout.** The [Web Scraper](https://scrape.ly/store/querystone/web-scraper.md) runs your own JavaScript page function on each page and follows the links you choose. This is the input we ran on 23 October 2026 against books.toscrape.com's mystery category, the same site as the ScrapeGraphAI example: ```json { "startUrls": [{"url": "https://books.toscrape.com/catalogue/category/books/mystery_3/index.html"}], "linkSelector": "article.product_pod h3 a", "globs": [{"glob": "https://books.toscrape.com/catalogue/*/index.html"}], "pageFunction": "async function pageFunction(context) {\n const $ = context.jQuery;\n if (!$('div.product_main').length) return null; // listing page: only follow links\n return {\n url: context.request.url,\n title: $('div.product_main h1').text().trim(),\n price: $('div.product_main p.price_color').text().trim(),\n inStock: $('div.product_main p.availability').text().includes('In stock'),\n upc: $('table.table-striped td').first().text().trim(),\n };\n}", "maxPagesPerCrawl": 6, "maxResultsPerCrawl": 5, "maxConcurrency": 2, "respectRobotsTxtFile": true, "proxyConfiguration": {"useScrapelyProxy": true} } ``` It returned four books in 12.4 seconds, among them "Sharp Objects" at £47.82 and "In a Dark, Dark Wood" at £19.63, each with its stock flag and UPC. The Web Scraper costs $0.715 per 1,000 results, platform usage included, and the selectors give the same answer on every run. If you'd rather not write selectors, ask an LLM to write the page function once, the way `ScriptCreatorGraph` does, and then run it at scale without a model. **3. Text for your own LLM.** When you do want a model to read the page (summaries, Q\&A, RAG), the [Website Content Crawler](https://scrape.ly/store/querystone/website-content-crawler.md) turns a site into clean Markdown for $0.50 per 1,000 pages plus platform usage, so you pay tokens only for the clean text. Our guide on [turning a website into Markdown for RAG](https://scrape.ly/blog/website-to-markdown-for-llms.md) has a real 10-page crawl and the chunking code. Large sites that block generic scrapers (Amazon, Google Maps, LinkedIn jobs, Zillow) have dedicated Actors in the [Store](https://scrape.ly/store.md), with the fields already mapped. ### Call it from Python or an agent Create an API token under **Settings → API & Integrations** and export it as `SCRAPELY_TOKEN`. The Python client (`pip install scrapely-client-python`) waits for the run and reads the rows: ```python import os from scrapely_client import ScrapelyClient client = ScrapelyClient(os.environ["SCRAPELY_TOKEN"], api_url="https://api.scrape.ly") run = client.actor("querystone/structured-data-extractor").call( run_input={ "startUrls": ["https://www.apple.com/shop/buy-iphone/iphone-16"], "onlyPagesWithStructuredData": True, } ) for row in client.dataset(run.default_dataset_id).iterate_items(): print(row["primaryType"], row["primaryName"], row.get("price"), row.get("priceCurrency")) ``` The prompt-style experience that makes ScrapeGraphAI appealing lives in your agent instead. Add the hosted [Scrape.ly MCP server](https://scrape.ly/mcp.md) to Claude Code with `claude mcp add --transport http scrapely https://mcp.scrape.ly`, sign in with `/mcp`, and ask: "Use scrapely to get the price and rating from these three product pages with querystone/structured-data-extractor." The agent picks the Actor, runs it and reads the result; the extraction itself costs no model tokens. ## ScrapeGraphAI vs Scrape.ly | | ScrapeGraphAI (library) | ScrapeGraphAI API | Scrape.ly | | --------------------------------- | ------------------------------ | ------------------------------------ | --------------------------------- | | LLM call per page | Yes, always | Yes, for Extract | No, for structured data | | Same page, same output | Depends on the model | Depends on the model | Yes | | 1,000 structured pages | Your LLM tokens + your servers | $10 (5 credits each on the $20 plan) | $1.11 (Structured Data Extractor) | | Browsers and proxies | You run them | Included | Included | | Ready-made scrapers for big sites | No | No | 570+ Actors in the Store | | Schedules, webhooks, Sheets, S3 | Your code | Monitor jobs | Built in | | Agent access | Through the API's MCP server | MCP server | Hosted MCP at `mcp.scrape.ly` | ScrapeGraphAI is a good way to see what an LLM can pull out of a page. For a pipeline you run every day, Scrape.ly is the better choice: deterministic output, a fixed price per row with no token bill, and the scheduling and delivery already built. Our overview of [AI web scraping](https://scrape.ly/blog/ai-web-scraping.md) compares rule-based, LLM and agent extraction, and the [Crawl4AI tutorial](https://scrape.ly/blog/crawl4ai-guide.md) covers the other popular open-source option. ## FAQ ### Is ScrapeGraphAI free? The library is open source under the MIT license, so the code is free. Every extraction calls an LLM, so you pay for model tokens (or run a local model through Ollama) plus your own servers. The hosted ScrapeGraphAI API has a one-time free allowance of 500 credits and paid plans from $20 a month. ### Does ScrapeGraphAI work without an OpenAI key? Yes, with a local model. Point the `llm` config at an Ollama model such as `ollama/llama3.2` and run an Ollama server. Without any model, the graph stops at the GenerateAnswer node, as our test showed. ### Which Python version does ScrapeGraphAI need? Version 2.3.1 requires Python 3.12 or newer. The `scrapegraph-py` SDK for the hosted API (2.1.0 and later) also needs 3.12. ### What is the difference between scrapegraphai and scrapegraph-py? `scrapegraphai` is the open-source library that runs graphs on your machine with your LLM. `scrapegraph-py` is the SDK for the paid hosted API, where ScrapeGraphAI runs the browser, proxies and model for you and bills in credits. ### What is a good ScrapeGraphAI alternative for production? Scrape.ly. Pages with schema.org markup go through the Structured Data Extractor ($1.11 per 1,000 pages), pages with a stable layout through the Web Scraper with your page function, and big sites through dedicated Actors, all without an LLM call per page. Agents run the same Actors over MCP. Scrape.ly Store Structured data without LLM tokens Price, rating, author, dates and more from any page's schema.org markup. $1.11 per 1,000 pages, platform usage included. [Try it free](https://scrape.ly/store/querystone/structured-data-extractor.md) [Browse the Store](https://scrape.ly/store.md) About the author [Scrape.ly Team](https://scrape.ly/blog/author/scrapely-team.md) The team behind Scrape.ly Scrape.ly is the web data platform from TeraShift GmbH in Switzerland, powered by Evomi: ready-made scrapers in the Scrape.ly Store, a cloud runtime, an API with JavaScript and Python SDKs, proxies, a hosted anti-bot browser and an MCP server for AI agents. - [Website](https://scrape.ly/index.md) Keep reading ## More in AI agents - [![Pixel-art robot with an orange >\_ on its terminal head carries a cable from a search results page headed by the Google G logo into a server rack labelled MCP](https://scrape.ly/media/web-search-mcp-server.png?w=750)](https://scrape.ly/blog/web-search-mcp-server.md) ### [6 Best Web Search MCP Servers for AI Agents in 2026](https://scrape.ly/blog/web-search-mcp-server.md) [The 6 best web search MCP servers for Claude, Cursor and ChatGPT, compared on where results come from, price per 1,000 searches, sign-in and spending control. With a real Google search run through MCP and the truth about Google Search MCP servers.](https://scrape.ly/blog/web-search-mcp-server.md) - [![Pixel-art robot with an orange >\_ on its terminal head carries a cable from a house, fronted by a red mailbox and a yard sign bearing the blue Zillow logo, into a server rack labelled MCP](https://scrape.ly/media/zillow-mcp.png?w=750)](https://scrape.ly/blog/zillow-mcp.md) ### [Zillow MCP: Property Data for Claude, ChatGPT and Cursor](https://scrape.ly/blog/zillow-mcp.md) [A Zillow MCP server lets your AI agent pull listings, prices, Zestimates and rent estimates into the chat as data it can compare and export. See how Zillow's own connector differs, connect Scrape.ly's hosted server to Claude, ChatGPT or Cursor, and see real tool calls, output and costs.](https://scrape.ly/blog/zillow-mcp.md) - [![Pixel-art robot with an orange >\_ on its terminal head, an AI sparkle in a speech bubble above it, steers a browser window with a game controller whose cable runs into the window, where a pointer clicks an orange button](https://scrape.ly/media/ai-browser-agents.png?w=750)](https://scrape.ly/blog/ai-browser-agents.md) ### [AI Browser Agents Explained: Browser Use, Computer Use and MCP](https://scrape.ly/blog/ai-browser-agents.md) [An AI browser agent is a language model that operates a web browser: it reads the page, picks an action, and clicks or types until the task is done. How Browser Use, Claude and OpenAI computer use and browser MCP servers work, where they struggle, and when a ready-made scraper is the better tool.](https://scrape.ly/blog/ai-browser-agents.md) ## The internet is full of data. Start owning yours. Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes. [Get Started Free](https://my.scrape.ly/register) [Chat with Support](mailto:hello@scrape.ly?subject=Question%20about%20Scrape.ly\&body=Hi%2C%20I%20have%20a%20question%20about%20Scrape.ly.) No credit card. No setup calls.