
Website Content Crawler
scrape.ly/store/querystone/website-content-crawlerAIDeveloper tools
1 monthly user

Extract Google AI Mode answers for any query: the full answer as text and Markdown, every cited source (title, URL, domain), product cards, follow-up questions and brand mentions for GEO monitoring. No login.
Get Google AI Mode answers for any question, at scale: the full answer as clean text and Markdown, every cited source with its real URL and domain, product cards, and the follow-up questions Google asks back. Add the brands and domains you care about and each row tells you whether AI Mode mentioned or cited them. No Google account or login needed.
| Field | Description |
|---|---|
queries | Questions or search queries, one per line. One row per query. |
brands | Brand or product names to look for in each answer. |
domains | Websites to look for among the cited sources, e.g. hubspot.com. |
countryCode | Two-letter country code, e.g. us, gb, de. Default us. |
languageCode | Google interface language, e.g. en, de. Default en. |
resolveSourceUrls | Resolve Google redirect links to real source URLs. Default on. |
maxConcurrency | Queries asked at the same time. Default 6. |
{
"queries": "What are the top 5 CRM tools\nbest running shoes for flat feet",
"brands": ["HubSpot", "Salesforce", "Pipedrive"],
"domains": ["hubspot.com"],
"countryCode": "us",
"languageCode": "en"
}
One row per query.
| Field | Description |
|---|---|
query | The query you sent. |
hasAnswer | Always true: only queries with an AI Mode answer are saved. |
text | The answer as plain text. |
markdown | The answer as Markdown, with [n] citation markers. |
sources | Cited sources: position, title, url, domain, siteName, description, citedInAnswer. |
sourcesCount / citedSourcesCount | Number of sources, and how many are cited inside the answer text. |
products | Product cards: title, price, merchant, rating, reviewsCount. |
followUpQuestions | Questions Google asks back at the end of the answer. |
mentionedBrands | Tracked brands found in the answer, in order of first mention. |
brandMentions | Per tracked brand: mentioned, count, rank, firstIndex. |
domainCitations | Per tracked domain: cited, citedInAnswer, positions. |
countryCode, languageCode, aiModeUrl, scrapedAt | Run context. |
{
"query": "best running shoes for flat feet",
"hasAnswer": true,
"text": "The best running shoes for individuals with flat feet include the Brooks Adrenaline GTS 25 for daily training, the ASICS Gel-Kayano 33 for premium traction and maximum stability, and the Saucony Hurricane 26 for highly cushioned comfort. ...",
"markdown": "The best running shoes for individuals with flat feet include the **Brooks Adrenaline GTS 25** for daily training, ... [1]\n\n### Top Running Shoes for Flat Feet\n\n**Brooks Adrenaline GTS 25** — $124.95, Naperville Running Company, 4.6/5 (1.6K reviews)\n- **Primary Use:** Daily training and everyday road runs.\n...",
"sources": [
{
"title": "7 Best Running Shoes For Flat Feet in 2026 | RunRepeat",
"url": "https://runrepeat.com/guides/best-flat-feet-running-shoes",
"domain": "runrepeat.com",
"siteName": "RunRepeat",
"description": "Compared to the average, the Hybrid ASICSGRIP rubber is 80% grippier, indicating all-weather performance and confidence. ...",
"citedInAnswer": true,
"position": 1
}
],
"sourcesCount": 12,
"citedSourcesCount": 3,
"products": [
{"title": "Brooks Adrenaline GTS 25", "price": "$124.95", "merchant": "Naperville Running Company", "rating": 4.6, "reviewsCount": "1.6K"}
],
"followUpQuestions": [
"What distances or weekly mileage do you typically run?",
"Do you also have a wider foot shape or encounter issues like toe rubbing?"
],
"mentionedBrands": ["Brooks", "ASICS"],
"countryCode": "US",
"languageCode": "en",
"aiModeUrl": "https://www.google.com/search?q=best+running+shoes+for+flat+feet&udm=50&hl=en&gl=us",
"scrapedAt": "2026-10-15T21:33:00Z"
}
You pay $0.15 per AI Mode answer (the ai-mode-answer event), plus the platform usage of the run (the hosted browser and compute).
Is it legal to scrape Google AI Mode? The actor reads answers that Google shows to any visitor without logging in. Check that your use of the data follows the laws and terms that apply to you.
Do I need a Google account or my own proxy? No. Google is opened in a hosted browser on a residential connection in the country you choose. The proxy setting is only a fallback for resolving source links.
Why does a query have no answer?
Google does not answer every query in AI Mode, and some topics are limited in some countries. Those queries are not saved as rows and are free: the run's status message and the failedInputs list in the OUTPUT record of the key-value store name each one with the reason.
Why is a source URL empty? When a Google redirect link cannot be resolved, the source keeps its title, site name and snippet. Turn on Resolve source URLs (the default) to fill URLs.
Can I schedule it or use it from my code? Yes. Schedule runs in the console, or start them through the API and read the dataset as JSON, CSV or Excel.
Are answers the same every time? No. AI Mode answers are generated and change between runs, by country and over time. That is what makes regular tracking useful.
languageCode, but brand matching is a plain text match.Every field below can be set in the run form, passed through the API, or saved as a task. Unset fields fall back to the defaults shown.
queriesstringRequiredQueries
Questions or search queries to ask Google AI Mode, one per line. Duplicates are removed. Each query returns one row.
brandsarrayOptionalBrands to track
Brand or product names to look for in each answer (GEO monitoring). Each row gets brandMentions (mentioned, count, rank by first mention) and mentionedBrands. Matching ignores case and whole words only.
domainsarrayOptionalDomains to track
Websites to look for among the cited sources, for example hubspot.com. Each row gets domainCitations: whether the domain is cited, cited inside the answer text, and at which source positions. Subdomains count.
countryCodestringOptionalCountry
Two-letter country code (us, gb, ca, au, in, de, ...). Google AI Mode is opened from a residential exit in this country, because answers differ by market. Default: us.
languageCodestringOptionalLanguage
Google interface language (hl), for example en, de, fr, es, pt-BR. Default: en.
resolveSourceUrlsbooleanOptionaldefault trueResolve source URLs
Follow Google's redirect links so each source has its real URL and domain. Turn off for slightly faster runs; sources then keep their title, site name and snippet, with the URL filled only where Google shows it.
maxConcurrencyintegerOptionaldefault 6Max concurrency
How many queries are asked at the same time.
proxyConfigurationobjectOptionaldefault {"useScrapelyProxy":true,"scrapelyProxyGroups":["DATACENTER…Proxy configuration
Proxy used only to resolve source redirect links that could not be resolved directly. Google AI Mode itself is opened in the hosted browser on a residential exit in the chosen country.
Every run writes its results to a dataset. Preview it in the dashboard, then export it or read it through the API.
| Field | Type | Description |
|---|---|---|
| query | string | |
| hasAnswer | boolean | |
| text | string | null | |
| markdown | string | null | |
| sources | array | |
| sourcesCount | number | |
| citedSourcesCount | number | |
| products | array | |
| followUpQuestions | array | |
| mentionedBrands | array | |
| brandMentions | array | |
| domainCitations | array | |
| countryCode | string | |
| languageCode | string | |
| aiModeUrl | string | |
| scrapedAt | string |
You pay for the events the actor reports, such as each result or page it produces. Platform usage is included in the event prices.
| Event | Charged when | Price |
|---|---|---|
| AI Mode answer | One query with a Google AI Mode answer (text, Markdown, sources, products, follow-ups). Queries without an answer are not charged. | $0.15 |
Run this actor from your own code. Swap in your API token and the examples work as-is.
import { ScrapelyClient } from 'scrapely-client'
const client = new ScrapelyClient({ token: process.env.SCRAPELY_TOKEN })
// Start the actor and wait for the run to finish.
const run = await client.actor('querystone/google-ai-mode-scraper').call({
"queries": "What are the top 5 CRM tools\nbest running shoes for flat feet\nhow does photosynthesis work\nbest project management software for small teams\nis intermittent fasting healthy\nbest VPN for streaming\nhow to start a podcast\nbest coffee grinder under 100\nwhat is a Roth IRA\nbest email marketing platform for ecommerce",
"brands": [
"HubSpot",
"Salesforce",
"Pipedrive"
],
"domains": [
"hubspot.com",
"salesforce.com"
],
"countryCode": "us",
"languageCode": "en",
"proxyConfiguration": {
"useScrapelyProxy": true,
"scrapelyProxyGroups": [
"DATACENTER"
]
}
})
// Read the dataset the run produced.
const { items } = await client.dataset(run.defaultDatasetId).listItems()
console.log(items)Tokens live in your account settings. Keep them server-side: a token carries the full permissions of your account.
Connect the Scrape.ly MCP server once and your agent can run this actor as a tool. Pick your tool:
Fetch scrape.ly/agents.md and run querystone/google-ai-mode-scraperPaste into Claude Code, Codex, Cursor or any AI agent. It signs you in once in the browser.
{
"mcpServers": {
"scrapely": {
"url": "https://api.scrape.ly/mcp?tools=actors,runs,storage,docs&actors=querystone/google-ai-mode-scraper"
}
}
}No key in the file: your client opens the browser to sign you in the first time it connects.
Inputs the developer prepared for common jobs. Open one to see its input, then copy it to your account to run it as-is or adjust it.
AI Mode answers with brand and domain mentions tracked.
Google AI Mode answers asked from the United Kingdom.
AI Mode answers in German with source links.
Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes. The advantage lasts forever.