
Website Content Crawler
scrape.ly/store/querystone/website-content-crawlerAIDeveloper tools
1 monthly user

Send queries to Microsoft Copilot Search in Bing and get the AI answer as Markdown, cited sources, inline citations, tables and brand mentions. Built for GEO and AI visibility tracking. No login needed.
Ask Microsoft Copilot Search (the AI answer mode in Bing) any number of questions and get each AI answer as clean Markdown, with the sources it cites, inline citations, tables and brand-mention stats. It's built for GEO and AI visibility tracking: see which brands Copilot recommends and which websites it trusts. You don't need a login or a Microsoft account, and you only pay for answers you get.
| Field | Description |
|---|---|
queries | Questions to ask, one per line. Duplicates are skipped. |
brands | Optional brand or product names to look for in each answer. |
countryCode | Country the questions are asked from (default us). |
maxConcurrency | How many queries run at the same time (default 5). |
proxy | Proxy settings. Datacenter is the default. A failing query switches to residential proxies in the chosen country automatically. |
{
"queries": "What are the top 5 CRM tools\nbest project management software for small teams",
"brands": ["HubSpot", "Salesforce", "Zoho"],
"countryCode": "us"
}
| Field | Description |
|---|---|
query | The question asked. |
text | The answer as clean Markdown (headings, bold, lists, tables), without citation markers. |
markdown | The same answer with [n] citation markers that point at sources[].id. |
tables | Tables in the answer: title, tableColumns, tableRows. |
sources | Sources with id, title, url, domain, sourceName, snippet and section (citations = cited in the answer, more = other pages Copilot read). |
inlineCitations | Each citation in the answer: id, textAnchor (the sentence it backs), title, url. |
sourcesCount, citationsCount, citedDomains | Quick counts and the list of cited domains. |
brandMentions, brandsMentioned | Per brand: mentioned, number of mentions, first position, rank and whether its site is cited. |
answerFromCache | true when Bing already had this answer ready. |
answerSeconds, countryCode, url, scrapedAt | Run details and the Copilot Search link. |
Example (trimmed):
{
"query": "What are the top 5 CRM tools",
"text": "The **top 5 CRM tools** most consistently ranked across expert sources are **monday.com, HubSpot CRM, Zoho CRM, Salesforce, and Klaviyo**, based on features, usability, and market presence.\n\n### 1. monday.com\n\n...",
"tables": [
{
"title": "Quick Comparison Table",
"tableColumns": ["CRM", "Best For", "Key Strengths", "Source"],
"tableRows": [[{"value": "monday.com"}, {"value": "Startups & new teams"}, {"value": "Ease of use, templates, automation"}, {"value": ""}]]
}
],
"sources": [
{
"id": 1,
"title": "10 Best CRM Software Of 2026 – Forbes Advisor",
"url": "https://www.forbes.com/advisor/business/software/best-crm-software/",
"domain": "forbes.com",
"sourceName": "Forbes",
"snippet": "Compare top CRM options to find the best fit for your needs and budget...",
"section": "citations"
}
],
"inlineCitations": [
{
"id": 1,
"textAnchor": "Forbes lists monday.com as the best CRM for beginners, highlighting its beautiful interface, intuitive tools, and ease of onboarding",
"title": "10 Best CRM Software Of 2026 – Forbes Advisor",
"url": "https://www.forbes.com/advisor/business/software/best-crm-software/"
}
],
"sourcesCount": 11,
"citationsCount": 3,
"citedDomains": ["forbes.com", "softwaretestinghelp.com", "emailanalytics.com"],
"brandsMentioned": ["HubSpot", "Zoho", "Salesforce"],
"brandMentions": [
{"brand": "HubSpot", "mentioned": true, "mentions": 7, "firstPosition": 89, "rank": 1, "cited": false, "citedUrls": []}
],
"countryCode": "us",
"answerFromCache": true,
"answerComplete": true,
"answerSeconds": 0.7,
"url": "https://www.bing.com/copilotsearch?q=What+are+the+top+5+CRM+tools",
"scrapedAt": "2026-10-15T21:42:03+00:00"
}
You pay $0.15 per answer ($150 per 1,000 answers), plus a little platform usage for compute and proxy traffic (on our test runs, well under $1 per 1,000 answers). So 100 answers cost about $15, and 1,000 answers about $150–151 in total. Queries that end without an answer cost nothing.
Is it legal? The actor reads the same public answers anyone sees on bing.com without logging in. Check that your use of the data follows the laws and terms that apply to you.
Do I need a login, cookies or a proxy? No login or cookies. Proxies are built in: datacenter by default, with an automatic switch to residential in your chosen country when a query fails.
Is this the same as the Copilot app? It's Copilot Search, the AI answer mode Microsoft runs in Bing (bing.com/copilotsearch). Like the Copilot app, it answers from live web results with citations, but it doesn't use the chat modes (Smart, Think Deeper, Study) of the signed-in app.
Why did a query get no answer? Copilot Search skips small talk and navigation queries (for example "hello" or a bare website name) and some sensitive topics. Those queries are listed in the run's OUTPUT record and aren't charged.
Why do answers differ between runs? AI answers change over time and by country. Run the same queries on a schedule to track the trend.
Can I schedule it or use the API? Yes. Schedule runs in the console, or start them and read the dataset through the REST API.
Every field below can be set in the run form, passed through the API, or saved as a task. Unset fields fall back to the defaults shown.
queriesstringRequiredQueries
The questions to ask Copilot Search, one per line. Each line becomes one result. Duplicates are skipped.
Track how often brands appear in Copilot's answers (GEO / AI visibility).
brandsarrayOptionalBrands to track
Optional. Brand or product names to look for in each answer. Every result then shows whether each brand is mentioned, how often, in which order and whether its website is cited.
countryCodeenumOptionalCountry
Country the questions are asked from. Copilot Search picks local sources, shops and prices for it. Default: United States.
maxConcurrencyintegerOptionalParallel queries
How many queries are asked at the same time. Higher is faster for long lists. Default: 5.
proxyobjectOptionaldefault {"useScrapelyProxy":true,"scrapelyProxyGroups":["DATACENTER…Proxy configuration
Proxies used to reach Copilot Search. Datacenter proxies work well; the actor switches a failing query to residential proxies in the chosen country automatically.
Every run writes its results to a dataset. Preview it in the dashboard, then export it or read it through the API.
| Field | Type | Description |
|---|---|---|
| query | string | The query sent to Copilot Search. |
| text | string | The answer as clean Markdown, without citation markers. |
| markdown | string | The answer as Markdown with [n] citation markers that point at sources[].id. |
| tables | array | Tables in the answer: title, column names and rows. |
| sources | array | Sources: section 'citations' (cited in the answer) or 'more' (other pages Copilot read). |
| inlineCitations | array | Each citation in the answer with the sentence it backs. |
| sourcesCount | number | |
| citationsCount | number | |
| citedDomains | array | |
| brandMentions | array | |
| brandsMentioned | array | |
| countryCode | string | |
| answerFromCache | boolean | |
| answerComplete | boolean | |
| answerSeconds | number | |
| url | string | |
| scrapedAt | string |
You pay for the events the actor reports, such as each result or page it produces. Platform usage is included in the event prices.
| Event | Charged when | Price |
|---|---|---|
| Copilot answer | One Copilot Search answer with its sources saved to the dataset. Queries without an answer are free. | $0.15 |
Run this actor from your own code. Swap in your API token and the examples work as-is.
import { ScrapelyClient } from 'scrapely-client'
const client = new ScrapelyClient({ token: process.env.SCRAPELY_TOKEN })
// Start the actor and wait for the run to finish.
const run = await client.actor('querystone/copilot-search-scraper').call({
"queries": "What are the top 5 CRM tools\nbest project management software for small teams",
"brands": [
"HubSpot",
"Salesforce",
"Zoho"
],
"proxy": {
"useScrapelyProxy": true,
"scrapelyProxyGroups": [
"DATACENTER"
]
}
})
// Read the dataset the run produced.
const { items } = await client.dataset(run.defaultDatasetId).listItems()
console.log(items)Tokens live in your account settings. Keep them server-side: a token carries the full permissions of your account.
Connect the Scrape.ly MCP server once and your agent can run this actor as a tool. Pick your tool:
Fetch scrape.ly/agents.md and run querystone/copilot-search-scraperPaste into Claude Code, Codex, Cursor or any AI agent. It signs you in once in the browser.
{
"mcpServers": {
"scrapely": {
"url": "https://api.scrape.ly/mcp?tools=actors,runs,storage,docs&actors=querystone/copilot-search-scraper"
}
}
}No key in the file: your client opens the browser to sign you in the first time it connects.
Inputs the developer prepared for common jobs. Open one to see its input, then copy it to your account to run it as-is or adjust it.
Copilot answers with cited sources.
Copilot answers with cited sources.
Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes. The advantage lasts forever.