
Website Content Crawler
scrape.ly/store/querystone/website-content-crawlerAIDeveloper tools
1 monthly user

Send queries to Google Gemini and get the AI answer as Markdown, cited sources, inline citations, tables and brand mentions. Built for GEO and AI visibility tracking. No login needed.
Ask Google Gemini any number of questions and get each answer as clean Markdown, with the web sources it cites, inline citations, tables and brand-mention stats. It's built for GEO and AI visibility tracking: see which brands Gemini recommends and which websites it trusts. You don't need a login or a Google account, answers usually arrive in under 10 seconds, and you only pay for answers you get.
| Field | Description |
|---|---|
queries | Questions to ask, one per line. Duplicates are skipped. |
brands | Optional brand or product names to look for in each answer. |
countryCode | Country the questions are asked from (default us). |
language | Interface language such as en, de or pt-br (default en). |
maxConcurrency | How many queries run at the same time (default 5). |
proxy | Proxy settings. A failing query switches to residential proxies in the chosen country automatically. |
{
"queries": "What are the top 5 CRM tools\nbest project management software for small teams",
"brands": ["HubSpot", "Salesforce", "Zoho"],
"countryCode": "us"
}
| Field | Description |
|---|---|
query | The question asked. |
text | Gemini's answer as Markdown (headings, bold, lists, tables). |
markdown | The same answer with [n] citation markers that point at sources[].id. |
tables | Tables in the answer: title, tableColumns, tableRows, source. When Gemini writes a comparison as headed sections with the same bold labels ("Best for:", "Pricing:") instead of a table, those sections come out as one table with source sections; real Markdown tables have source table. |
sources | Sources with id, title, url, domain, sourceName, snippet and section (citations = cited in the answer, more = other pages from Gemini's web search). |
inlineCitations | Each citation: id, textAnchor (the sentence it backs), title, url and textFragmentUrl (the link that highlights the quoted passage). |
sourcesCount, citationsCount, citedDomains | Quick counts and the list of cited domains. |
brandMentions, brandsMentioned | Per brand: mentioned, number of mentions, first position, rank and whether its site is cited. |
model, answerSeconds, countryCode, language, scrapedAt | Which Gemini model answered, and run details. |
Example (trimmed):
{
"query": "What are the top 5 CRM tools",
"text": "Here are the top 5 CRM (Customer Relationship Management) tools, widely recognized for their features, usability, and market share:\n\n### 1. Salesforce (Sales Cloud)\n* **Best for:** Mid-market to enterprise-level businesses requiring deep customization.\n...",
"tables": [],
"sources": [
{
"id": 1,
"title": "10 Best CRM Tools for Sales Teams in 2026 - Sybill",
"url": "https://www.sybill.ai/blogs/best-crm-tools",
"domain": "sybill.ai",
"sourceName": "Sybill",
"snippet": "10 Best CRM Tools for Sales Teams in 2026 * 1. Salesforce. Best for: Mid-market to enterprise teams that need deep customization...",
"section": "citations"
}
],
"inlineCitations": [
{
"id": 1,
"textAnchor": "Mid-market to enterprise-level businesses requiring deep customization.",
"title": "10 Best CRM Tools for Sales Teams in 2026 - Sybill",
"url": "https://www.sybill.ai/blogs/best-crm-tools",
"textFragmentUrl": "https://www.sybill.ai/blogs/best-crm-tools#:~:text=10%20Best%20CRM%20Tools%20for%20Sales%20Teams"
}
],
"sourcesCount": 8,
"citationsCount": 3,
"citedDomains": ["sybill.ai", "solutionsreview.com", "ringover.com"],
"brandsMentioned": ["Salesforce", "HubSpot", "Zoho"],
"brandMentions": [
{"brand": "HubSpot", "mentioned": true, "mentions": 2, "firstPosition": 628, "rank": 2, "cited": false, "citedUrls": []}
],
"countryCode": "us",
"language": "en",
"model": "3.5 Flash-Lite",
"answerSeconds": 4.3,
"scrapedAt": "2026-10-15T22:07:07+00:00"
}
You pay $0.15 per answer ($150 per 1,000 answers), plus a little platform usage for compute and proxy traffic (on our test runs, well under $1 per 1,000 answers). So 100 answers cost about $15, and 1,000 answers about $150–151 in total. Queries that end without an answer cost nothing.
Is it legal? The actor reads the same answers anyone gets on gemini.google.com without signing in. Check that your use of the data follows the laws and terms that apply to you.
Do I need a login, cookies or a proxy? No login or cookies. Proxies are built in, with an automatic switch to residential proxies in your chosen country when a query fails.
Which Gemini model answers? The model Google gives signed-out visitors (currently Gemini 3.5 Flash-Lite, with Google Search). Every row says which model answered. Paid Gemini models need a signed-in account and aren't used.
Does Gemini always search the web? Gemini decides per question. Most product and research questions trigger a Google Search, and those answers carry citations. Small talk and simple facts often don't.
Why do answers differ between runs? AI answers change over time and by country. Run the same queries on a schedule to track the trend.
Can I schedule it or use the API? Yes. Schedule runs in the console, or start them and read the dataset through the REST API.
Every field below can be set in the run form, passed through the API, or saved as a task. Unset fields fall back to the defaults shown.
queriesstringRequiredQueries
The questions to ask Gemini, one per line. Each line becomes one result. Duplicates are skipped.
Track how often brands appear in Gemini's answers (GEO / AI visibility).
brandsarrayOptionalBrands to track
Optional. Brand or product names to look for in each answer. Every result then shows whether each brand is mentioned, how often, in which order and whether its website is cited.
countryCodeenumOptionalCountry
Country the questions are asked from. Gemini picks local sources, shops and prices for it. Default: United States.
languagestringOptionalInterface language
Language code of the Gemini interface, such as en, de, fr, es or pt-br. Gemini usually answers in the language of the question. Default: en.
maxConcurrencyintegerOptionalParallel queries
How many queries are asked at the same time. Higher is faster for long lists. Default: 5.
proxyobjectOptionaldefault {"useScrapelyProxy":true,"scrapelyProxyGroups":["DATACENTER…Proxy configuration
Proxies used to reach Gemini. A failing query switches to residential proxies in the chosen country automatically.
Every run writes its results to a dataset. Preview it in the dashboard, then export it or read it through the API.
| Field | Type | Description |
|---|---|---|
| query | string | The query sent to Gemini. |
| text | string | The answer as clean Markdown, without citation markers. |
| markdown | string | The answer as Markdown with [n] citation markers that point at sources[].id. |
| tables | array | Tables in the answer: title, column names and rows. |
| sources | array | Sources: section 'citations' (cited in the answer) or 'more' (other pages Gemini found in its web search). |
| inlineCitations | array | Each citation in the answer with the sentence it backs and the highlight link Gemini uses. |
| sourcesCount | number | |
| citationsCount | number | |
| citedDomains | array | |
| brandMentions | array | |
| brandsMentioned | array | |
| countryCode | string | |
| answerSeconds | number | |
| scrapedAt | string | |
| language | string | |
| model | string | The Gemini model that answered. |
You pay for the events the actor reports, such as each result or page it produces. Platform usage is included in the event prices.
| Event | Charged when | Price |
|---|---|---|
| Gemini answer | One Gemini answer with its sources saved to the dataset. Queries without an answer are free. | $0.15 |
Run this actor from your own code. Swap in your API token and the examples work as-is.
import { ScrapelyClient } from 'scrapely-client'
const client = new ScrapelyClient({ token: process.env.SCRAPELY_TOKEN })
// Start the actor and wait for the run to finish.
const run = await client.actor('querystone/gemini-search-scraper').call({
"queries": "What are the top 5 CRM tools\nbest project management software for small teams",
"brands": [
"HubSpot",
"Salesforce",
"Zoho"
],
"proxy": {
"useScrapelyProxy": true,
"scrapelyProxyGroups": [
"DATACENTER"
]
}
})
// Read the dataset the run produced.
const { items } = await client.dataset(run.defaultDatasetId).listItems()
console.log(items)Tokens live in your account settings. Keep them server-side: a token carries the full permissions of your account.
Connect the Scrape.ly MCP server once and your agent can run this actor as a tool. Pick your tool:
Fetch scrape.ly/agents.md and run querystone/gemini-search-scraperPaste into Claude Code, Codex, Cursor or any AI agent. It signs you in once in the browser.
{
"mcpServers": {
"scrapely": {
"url": "https://api.scrape.ly/mcp?tools=actors,runs,storage,docs&actors=querystone/gemini-search-scraper"
}
}
}No key in the file: your client opens the browser to sign you in the first time it connects.
Inputs the developer prepared for common jobs. Open one to see its input, then copy it to your account to run it as-is or adjust it.
Gemini answer to a US question with brand mentions tracked.
Gemini answers for several questions asked from Germany.
Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes. The advantage lasts forever.