--- title: "How to Scrape Google AI Overviews (2026) | Scrape.ly" description: "Use a Google AI Overview scraper to get overview text and cited sources per keyword, plus AI Mode answers, with real runs, costs and code for Python." url: https://scrape.ly/blog/scrape-google-ai-overviews markdown: https://scrape.ly/blog/scrape-google-ai-overviews.md --- [Tutorials](https://scrape.ly/blog/category/tutorials.md) # How to Scrape Google AI Overviews and AI Mode Answers Check which keywords trigger a Google AI Overview and get the answer text with every cited source, by country and language. Add AI Mode answers with brand and domain tracking. Real runs, real costs, and code for curl, Python, Node.js and AI agents. [Querystone Actor publisher on the Scrape.ly Store](https://scrape.ly/blog/author/querystone.md) - Published Oct 23, 2026 - Reading time 9 min read ![Pixel-art robot with an orange >\_ on its terminal head lifts an AI overview card marked with an orange sparkle off a search page headed by the Google G logo, where an AI answer panel sits above the results, and drops it into a crate](https://scrape.ly/media/scrape-google-ai-overviews.png?w=1080) To scrape Google AI Overviews, send your keywords to a Google AI Overview scraper that opens Google in a real browser from the country you choose and saves one row per keyword: whether an overview appeared, its full text and every source it cites. On Scrape.ly that's the Google AI Overviews Scraper, at $1 per 1,000 overviews, and keywords without an overview cost no event fee. For Google's separate AI Mode tab, a second Actor returns the full answer with sources, follow-up questions and brand mentions. This guide covers the fields, the no-code steps, the API (curl, Python, Node.js and MCP), costs and pitfalls. It uses two real runs from 23 October 2026: three US keywords for AI Overviews in 26 seconds, and one AI Mode answer with brand tracking. ## What data you can get from Google AI Overviews The [Google AI Overviews Scraper](https://scrape.ly/store/querystone/google-ai-overviews-scraper.md) writes **one row per keyword**: | Field | What it holds | | --------------- | --------------------------------------------------------------------------------------------------- | | `query` | The keyword you entered | | `hasAiOverview` | `true` when Google showed an AI Overview for it | | `type` | `static` when the overview came with the results page, `live` when Google streamed it in afterwards | | | | | | ------ | --------------------------------------------------------------------------------------------------------------- | -- | | `text` | The overview answer, one paragraph or list item per line; list items start with \`- \` and table rows use \` \\ | \` | | | | | ----------------------------- | ------------------------------------------------------------------------------------------------ | | `sources[]` | Every cited source: `position`, `url`, `title`, `description`, `source` (site name) and `domain` | | `sourcesCount` | Number of cited sources | | `countryCode`, `languageCode` | The market the keyword was checked in | | `searchUrl`, `scrapedAt` | The Google results URL and the time of the check (UTC) | | `error` | Only on rows that could not be checked, never charged | Source cards come first in `sources`, then sites linked only from inside the answer. The text behind Google's "Show more" is included; images, videos and widgets inside an overview are not. ## How to run the Google AI Overview scraper without code 1. Open the Actor in the Scrape.ly Store and sign in. 2. Pick the preset [AI Overviews for 3 US searches](https://scrape.ly/store/querystone/google-ai-overviews-scraper/presets/search-us.md). "Try this input" opens it in the console. 3. Check the input. This is the exact JSON of our run: ```json { "queries": "what is python programming\nhow to choose a heat pump\nkubernetes pros and cons", "countryCode": "us", "languageCode": "en", "includeQueriesWithoutOverview": false } ``` 4. Click **Start**. Our run took 26 seconds. 5. Open the **AI Overviews** view for one row per keyword, or **Cited sources** for one row per source, and export as JSON, CSV, Excel or HTML. The fields that matter most: - **`queries`** (required): one keyword per line, up to 5,000 per run. Duplicates are removed. - **`countryCode`** and **`languageCode`**: the market (default `us`, `en`). Google is opened from a residential exit in that country, because overviews differ by market. The preset [AI Overviews for 3 German searches](https://scrape.ly/store/querystone/google-ai-overviews-scraper/presets/search-3.md) shows another market. - **`includeQueriesWithoutOverview`**: on by default. It saves a free row with `hasAiOverview: false` for every keyword without an overview, which is how you measure coverage. Turn it off to keep only overviews. Scrape.ly Store Google AI Overviews Scraper One row per keyword: whether Google showed an AI Overview, the full answer text and every cited source, in any country. $1 per 1,000 overviews; keywords without one cost no event fee. [Try it free](https://scrape.ly/store/querystone/google-ai-overviews-scraper.md) [Browse the Store](https://scrape.ly/store.md) ## How to scrape Google AI Overviews with the API Create an API token under **Settings → API & Integrations** and export it as `SCRAPELY_TOKEN`. **curl.** Our three keywords took 26 seconds, and the README puts 10 keywords at about 45 seconds, the limit of the synchronous endpoint. Use the asynchronous flow: start the run, wait, then read the rows. ```shell # 1. start the run; the response's data.id is the run ID curl -X POST "https://api.scrape.ly/v2/actors/querystone~google-ai-overviews-scraper/runs" -H "Authorization: Bearer $SCRAPELY_TOKEN" -H "Content-Type: application/json" -d '{"queries": "what is python programming\nhow to choose a heat pump\nkubernetes pros and cons", "countryCode": "us", "languageCode": "en", "includeQueriesWithoutOverview": false}' # 2. wait (up to 60 s per call; repeat until status is SUCCEEDED) curl "https://api.scrape.ly/v2/actor-runs/RUN_ID?waitForFinish=60" -H "Authorization: Bearer $SCRAPELY_TOKEN" # 3. read the rows (CSV here) curl "https://api.scrape.ly/v2/actor-runs/RUN_ID/dataset/items?format=csv" -H "Authorization: Bearer $SCRAPELY_TOKEN" ``` **Python** (`pip install scrapely-client-python`). `call()` waits however long the run takes. This example counts how often each domain is cited across your keywords, the core number of AI search visibility: ```python import os from collections import Counter from scrapely_client import ScrapelyClient client = ScrapelyClient(os.environ["SCRAPELY_TOKEN"], api_url="https://api.scrape.ly") run = client.actor("querystone/google-ai-overviews-scraper").call( run_input={"queries": "what is python programming\nhow to choose a heat pump\nkubernetes pros and cons", "countryCode": "us", "languageCode": "en"} ) print(run.status) # SUCCEEDED, FAILED, ABORTED or TIMED-OUT cited = Counter() for row in client.dataset(run.default_dataset_id).iterate_items(): print(row["query"], row["hasAiOverview"], row["sourcesCount"]) cited.update({s["domain"] for s in row["sources"]}) print(cited.most_common(10)) ``` **Node.js** (`npm install scrapely-client`): ```javascript import { ScrapelyClient } from 'scrapely-client'; const client = new ScrapelyClient({ token: process.env.SCRAPELY_TOKEN, baseUrl: 'https://api.scrape.ly' }); const run = await client.actor('querystone/google-ai-overviews-scraper').call({ queries: 'what is python programming\nhow to choose a heat pump\nkubernetes pros and cons', countryCode: 'us', languageCode: 'en', }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items.map((r) => [r.query, r.hasAiOverview, r.sources.map((s) => s.domain).join(', ')])); ``` **AI agents (MCP).** Add the hosted Scrape.ly MCP server. In Cursor, put this in `~/.cursor/mcp.json`: ```json { "mcpServers": { "scrapely": { "url": "https://api.scrape.ly/mcp", "headers": { "Authorization": "Bearer YOUR_TOKEN" } } } } ``` In Claude Code, run `claude mcp add --transport http scrapely https://api.scrape.ly/mcp` and sign in with `/mcp`. Then ask: "Use scrapely to run querystone/google-ai-overviews-scraper for my 20 keywords in the US and tell me which domains are cited most and whether mysite.com appears." Runs longer than 45 seconds keep going; the agent polls `get-actor-run`, then reads the rows with `get-actor-output`. The [Google AI Overviews API](https://scrape.ly/api/google-ai-overviews-api.md) page shows the same Actor as an API product. ## A real run: three US keywords We ran the preset on 23 October 2026. All three keywords had an AI Overview. The run saved 3 rows in 26 seconds and was charged $0.0259. | Keyword | Type | Sources | Top cited domains | | -------------------------- | ------ | ------- | -------------------------------------------- | | how to choose a heat pump | static | 10 | reddit.com, nytimes.com, consumerreports.org | | what is python programming | live | 7 | reddit.com, python.org, coursera.org | | kubernetes pros and cons | live | 10 | dev.to, altexsoft.com, reddit.com | What the rows show: - **Reddit is everywhere.** reddit.com was cited for all three keywords, first for two of them. YouTube was cited twice for both the heat pump and the Python query. - **Two of three overviews were streamed in.** Google added them after the page loaded (`type: live`). A plain HTTP scraper that reads only the first HTML would have missed both. - **The answers are long.** The overview texts ran from 2,596 to 3,466 characters, with headings and bullet lists kept as lines. - **Official sources still count.** python.org appeared twice in the Python overview's sources, as a source card and as a link inside the answer. The log, without the platform's start-up lines: ```plaintext 20261023T03:02:32.904Z [scrapely.actor] INFO Checking 3 queries for AI Overviews (country=US, language=en, concurrency=3). 20261023T03:02:38.040Z [scrapely.actor] INFO browser_session attempt=1 state=sorry latency_ms=4644 connect=1574 commit=2819 ready=4533 20261023T03:02:41.983Z [scrapely.actor] INFO browser_session attempt=2 state=serp latency_ms=3835 connect=1508 commit=2711 ready=3707 20261023T03:02:41.984Z [scrapely.actor] INFO session_ready live=1 after_ms=9011 20261023T03:02:49.042Z [scrapely.actor] INFO Done: 3 AI Overviews, 0 queries without one, 0 failed, 33 queries in 16 s. (deferred overviews: 2, deferred without overview: 0, partial: 0) 20261023T03:02:49.694Z [Scrapely] Container state: status=exited exitCode=0 oomKilled=false ``` The first browser session landed on Google's "sorry" page, its block page for automated traffic. The Actor opened a second session, got a normal results page, and read all three overviews in 16 seconds. "deferred overviews: 2" are the two `live` rows. ### Plus one Google AI Mode answer AI Mode is a separate tab with chat-style answers, and the AI Overviews Actor doesn't cover it. The [Google AI Mode Scraper](https://scrape.ly/store/querystone/google-ai-mode-scraper.md) does. We asked it one question from its preset [2 CRM questions with brand tracking](https://scrape.ly/store/querystone/google-ai-mode-scraper/presets/query-brand-tracking.md): ```json { "queries": "best crm for startups", "brands": ["HubSpot", "Salesforce", "Pipedrive"], "domains": ["hubspot.com"], "countryCode": "us", "languageCode": "en" } ``` The answer came back in 12 seconds with 20 sources, 10 of them cited inside the answer text. The brand fields did the counting: `mentionedBrands` listed HubSpot (5 mentions, first) and Pipedrive (3 mentions, second). Salesforce wasn't named in the answer, although salesforce.com was cited as source 4. `domainCitations` showed hubspot.com cited at positions 15 and 17, both inside the answer. The row also carried 3 follow-up questions Google asked back and no product cards. The answer named Attio, Zoho CRM and Close as well, so add every competitor you care about to `brands`. ## What it costs **AI Overviews:** one event, $0.001 per keyword with an overview, so $1.00 per 1,000 overviews. Keywords without an overview and failed checks pay no event fee. Platform usage (the hosted browser that opens Google, plus compute) is billed on top. For our run: 3 overviews × $0.001 = $0.003 in event fees. The run was charged $0.0259 in total: $0.021 of hosted browser time, $0.0017 of compute and $0.0002 of proxy traffic and storage. Browser time weighs more on a tiny run like this one ($0.007 per keyword). In the publisher's own test of 1,000 keywords, usage came to about $2.10, and the run totaled about $3.08 with $0.98 in events. **AI Mode:** $0.15 per answer, plus usage. Our single answer was charged $0.1746: $0.15 for the answer, $0.024 of browser time and under $0.001 of compute and storage. Queries without an AI Mode answer are free and aren't saved. Set a maximum charge per run for a hard cap; both Actors stop cleanly before they would go over it. ## What to watch out for when scraping AI Overviews - **Overviews come and go.** Google shows one only when it decides it adds something. Navigational, local and very short keywords rarely get one, and the same keyword can get one today and not tomorrow. Track coverage with the free `hasAiOverview: false` rows instead of assuming every keyword has an answer. - **Many overviews load late.** Two of our three were streamed in after the page loaded. The Actor waits for them; DIY scrapers that read the first HTML miss them. - **Google blocks automated sessions.** Our log shows a "sorry" page on the first attempt. The Actor retries on a fresh session, so you don't write that logic. - **Answers differ by market and over time.** Overviews vary with location, language and day. Fix `countryCode` and `languageCode` per market and schedule the same list weekly to see real changes. - **AI Mode is a different product.** Our AI Mode answer cited 20 sources against 7 to 10 for the overviews, and it's priced per answer. Use it for the questions where buyers ask Google for recommendations, and AI Overviews for broad keyword coverage. ## Is it legal to scrape Google AI Overviews? The Actors read the same public results page any logged-out visitor sees, without a Google account. Google's rules on automated access are specific, though. Its [Terms of Service](https://policies.google.com/terms) list "using automated means to access content from any of our services in violation of the machine-readable instructions on our web pages (for example, robots.txt files that disallow crawling, training, or other activities)" as abuse, and Google's [robots.txt](https://www.google.com/robots.txt) disallows `/search` for all crawlers. Read both before you build a product on this data. Overview text is generated by Google and summarizes publishers' pages, so use it for analysis rather than republishing it. Overviews rarely contain personal data; if a source or answer names a person and you store it, the GDPR and CCPA apply. For visibility research on public results at modest volumes, those are the habits that matter. This is general information, not legal advice: for commercial use, ask a lawyer about your case. ## Why Scrape.ly for AI Overview data - **Google only reports your own clicks.** Google's own guidance says sites appearing in AI Overviews and AI Mode are included in Search Console's overall traffic, reported within the "Web" search type ([Google Search Central](https://developers.google.com/search/docs/appearance/ai-features)). That covers only your own site and doesn't show the answer or who else was cited. - **No selectors to babysit.** Hand-written guides drive Selenium, click "Show more" and parse Google's generated class names, then add retries for missing overviews. The Actor returns `text` and `sources` as fields and handles late-loading overviews and block pages. - **You pay for answers, not attempts.** $1 per 1,000 overviews, nothing for keywords without one, plus a maximum charge per run. - **One platform for all AI search.** Pair this Actor with the AI Mode Scraper, the [ChatGPT Search Scraper](https://scrape.ly/store/querystone/chatgpt-search-scraper.md) and the Google Search Scraper's [live AI overview preset](https://scrape.ly/store/querystone/google-search-scraper/presets/live-ai-overview-ranking-check.md), which puts the overview next to organic rankings. Our [AI search visibility tracking](https://scrape.ly/blog/ai-search-visibility-tracking.md) guide turns these into a weekly brand report, and [scraping Google search results](https://scrape.ly/blog/scrape-google-search-results.md) covers the classic blue links. ## FAQ ### Does Google offer an API for AI Overviews? Google's documentation for site owners points to Search Console, which counts traffic from AI features inside its Web search totals for your own site only; it doesn't return the overview text or the other cited sites. To get the overview text and cited sources for any keyword, use a scraper such as the Google AI Overviews Scraper, which you can also call as a REST API. ### How do I know which keywords trigger an AI Overview? Run your keyword list with `includeQueriesWithoutOverview` on (the default). Every keyword gets a row, and `hasAiOverview` is `true` or `false`. Rows without an overview carry no event fee, so measuring coverage is cheap. ### Can I scrape Google AI Mode answers too? Yes, with the Google AI Mode Scraper. It returns the answer as text and Markdown, every source with its real URL and whether it's cited inside the answer, product cards, follow-up questions, and mentions of the brands and domains you list. It costs $0.15 per answer. ### How much does it cost to scrape Google AI Overviews? $1.00 per 1,000 overviews in event fees, plus hosted-browser usage. Our 3-keyword run cost $0.0259 in total; the publisher measured about $3.08 for 1,000 keywords. Keywords without an overview pay no event fee. ### Can I scrape AI Overviews for another country? Yes. Set `countryCode` (for example `gb`, `de` or `in`) and `languageCode`. The Actor opens Google from a residential exit in that country, so you see the overview local users see. Scrape.ly Store Track your keywords in AI Overviews Paste your keyword list, pick a country, and get the overview text and every cited source per keyword. $1 per 1,000 overviews. [Try it free](https://scrape.ly/store/querystone/google-ai-overviews-scraper.md) [Browse the Store](https://scrape.ly/store.md) About the author [Querystone](https://scrape.ly/blog/author/querystone.md) Actor publisher on the Scrape.ly Store Search results, AI answers, web pages and market feeds as structured data. For SEO teams, developers and AI builders, plus finance, crypto and sports data for analysts. A Scrapely studio. - [Website](https://scrape.ly/users/querystone.md) Keep reading ## More in Tutorials - [![Pixel-art robot with an orange >\_ on its terminal head takes a classified ad slip with a purple title off a cork board topped with the purple Craigslist peace-sign logo and drops it into a crate](https://scrape.ly/media/scrape-craigslist.png?w=750)](https://scrape.ly/blog/scrape-craigslist.md) ### [How to Scrape Craigslist Listings: Prices, Locations and Full Ads](https://scrape.ly/blog/scrape-craigslist.md) [Scrape Craigslist by search URL or by city and category and get every ad as a row: title, price, neighborhood, category, posting date, attributes and the full description. A real run on New York bike ads, its cost, code for curl, Python and Node.js, and how to use the data responsibly.](https://scrape.ly/blog/scrape-craigslist.md) - [![Pixel-art robot with an orange >\_ on its terminal head pulls an ad card with an image and a blue button out of a monitor showing the blue Facebook logo above two more ad cards, and drops it into a crate](https://scrape.ly/media/scrape-facebook-ads-library.png?w=750)](https://scrape.ly/blog/scrape-facebook-ads-library.md) ### [How to Scrape the Facebook Ads Library (Meta Ad Library)](https://scrape.ly/blog/scrape-facebook-ads-library.md) [Scrape the Facebook Ads Library for any brand, page or keyword: ad copy, creatives, call to action, landing page, start date, platforms and the Ad Library link. A real run on Nike's active US ads, its exact cost, and code for curl, Python, Node.js and AI agents.](https://scrape.ly/blog/scrape-facebook-ads-library.md) - [![Pixel-art robot with a terminal for a head pulling a place card out of a Google Maps pin standing on a folded map, with a crate of collected place cards beside it](https://scrape.ly/media/how-to-scrape-google-maps.png?w=750)](https://scrape.ly/blog/how-to-scrape-google-maps.md) ### [How to Scrape Google Maps: Places, Ratings, Phones and Websites](https://scrape.ly/blog/how-to-scrape-google-maps.md) [Scrape Google Maps places for any search term and city, with no code or with one API call. A real run, its output, its log and its exact cost, plus the limits and the legal side.](https://scrape.ly/blog/how-to-scrape-google-maps.md) ## The internet is full of data. Start owning yours. Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes. The advantage lasts forever. [Get Started Free](https://my.scrape.ly/register) [Chat with Support](mailto:support@scrape.ly?subject=Question%20about%20Scrape.ly\&body=Hi%2C%20I%20have%20a%20question%20about%20Scrape.ly.) No credit card. No setup calls.