Google Search Scraper

🔍 Google Search Scraper

Turn a list of Google queries into dataset pages of organic results, People Also Ask, and related searches, with ads and AI overviews when you ask for them.

For SEO, brand monitoring, and research jobs that need the public results page; a page that fails is reported in the run's status and OUTPUT record instead of ending the run.

✨ Features

• ⚡ Fast search mode by default. Result pages come back in about a second each, with no browser to start. Options that need the full page switch to browser mode on their own.
• 📄 One item per page. Each query line and each page is its own dataset row, with the query, the results URL, and the country and language that were sent.
• 🌐 Organic and paid. Titles, URLs, displayed URLs, and snippets. Turn on Include ads to store ads with their ad position.
• ❓ Page modules. People Also Ask and related queries, plus the AI overview in browser mode.
• 🎯 Query operators. Site, related site, intitle, intext, inurl, file type, exact match, date range, and a results language.
• 📎 Redirects. Google /goto result links are followed one hop so the dataset stores the destination when that hop succeeds.
• 💾 HTML. Optional raw HTML on the item, and an HTML snapshot in the key-value store.

🚀 Why teams use this

• Built for a scheduled production run: many queries, a concurrency cap, and a page that failed listed in the status message and the OUTPUT record (failedInputs) while the other queries keep going.
• Empty strings and empty lists stay off the item. organicResults is an empty list when the page was not a results page, so a failure is visible.
• A later page is not fetched once the page has no organic results or hasNextPage is false.
• If fast search mode can't return a page, that page is loaded in browser mode and the run carries on.

⚡ Fast search mode and browser mode

Runs use fast search mode unless an option needs the full results page. These options use browser mode for the whole run:

• Include ads (includeAds)
• Wait for live AI overviews (liveAiOverview)
• Ranking check (webTabRankingCheck)
• Save HTML to dataset (saveHtml)
• Save HTML snapshots (saveHtmlToKeyValueStore)
• Exact location (locationUule)
• Unfiltered results (includeUnfilteredResults)

A query line that is a full Google Search URL also uses browser mode. Everything else, including the query operators, date filters, country, interface language, search language, results per page, and pages per query, works in fast search mode.

Google results pages hold 10 results. In fast search mode, a Results per page other than 10 is filled from as many consecutive Google pages as it needs and trimmed to size, so 20 results per page takes 2 Google pages. Browser mode loads one Google page per item and sends the page size as num.

If fast search mode fails for a page (an error, a timeout, or no results on a first page), that page is loaded in browser mode instead and the log says Switching to browser mode for this query page. The next page tries fast search mode again.

Fast search mode fills the same fields as browser mode, except for these, which only browser mode fills: paidResults, aiOverview, resultsTotal, html, htmlSnapshotUrl, and on organic rows websiteTitle and emphasizedKeywords. Fields a mode can't fill are left off the item, as with any empty field.

💰 Pricing

Pay per event: SERP page, $3.00 per 1,000 pages ($0.003 each). One event is charged for each Google results page retrieved, in either mode. With the default 10 results per page that's one event per dataset item. In fast search mode a larger page size retrieves more Google pages (20 results per page is 2 events per item); a Google page past the last result isn't charged. Failed pages are not saved and not charged; they are listed under failedInputs in the OUTPUT record. The run checks its remaining budget before each page and stops cleanly when the next page would go over its maximum total charge.

💡 Use cases

• Track keyword rankings for your site and your competitors by country.
• Collect People Also Ask questions and related searches for content research.
• Monitor which ads and AI Overviews appear for your brand terms.
• Feed fresh search results into an LLM or research agent.

🧪 Example input

{
  "countryCode": "us",
  "maxPagesPerQuery": 1,
  "queries": "web scraping",
  "resultsPerPage": 10
}

🧾 Sample output

One item from a real run of the example input in fast search mode, shortened. Long text is cut and lists show their first entries. With Include ads on, the run uses browser mode and the item also has paidResults.

{
  "searchQuery": {
    "term": "web scraping",
    "url": "https://www.google.com/search?q=web+scraping&hl=en&num=10&gl=us",
    "device": "DESKTOP",
    "page": 1,
    "type": "SEARCH",
    "domain": "google.com",
    "countryCode": "US",
    "languageCode": "en",
    "resultsPerPage": 10
  },
  "url": "https://www.google.com/search?q=web+scraping&hl=en&num=10&gl=us",
  "organicResults": [
    {
      "title": "Web scraping",
      "url": "https://en.wikipedia.org/wiki/Web_scraping",
      "type": "organic",
      "displayedUrl": "https://en.wikipedia.org › wiki › Web_scraping",
      "description": "Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites.",
      "position": 1
    },
    {
      "title": "What is Web Scraping and How to Use It?",
      "url": "https://www.geeksforgeeks.org/blogs/what-is-web-scraping-and-how-to-use-it/",
      "type": "organic",
      "displayedUrl": "https://www.geeksforgeeks.org › blogs › what-is-web-scraping-and-how-to-use-it",
      "description": "Web scraping is an automated method to extract large amounts of data from websites. This data, usually in HTML format, is converted into ...",
      "date": "2025-11-12T12:00:00.000Z",
      "position": 2
    }
  ],
  "peopleAlsoAsk": [
    {
      "question": "Can ChatGPT do web scraping?",
      "title": "Can ChatGPT do web scraping?"
    },
    {
      "question": "What actually is web scraping?",
      "title": "What actually is web scraping?"
    }
  ],
  "relatedQueries": [
    {
      "title": "Web scraping tools",
      "url": "https://www.google.com/search?q=Web+scraping+tools&hl=en&gl=us"
    },
    {
      "title": "Web scraping python",
      "url": "https://www.google.com/search?q=Web+scraping+python&hl=en&gl=us"
    }
  ],
  "hasNextPage": true
}

📥 Input

queries is one query or URL per line. maxPagesPerQuery and resultsPerPage set the page size. countryCode is gl. languageCode is hl. searchLanguage is lr. locationUule is passed through as uule.

Operators: forceExactMatch, site, relatedToSite, wordsInTitle, wordsInText, wordsInUrl, fileTypes, quickDateRange, beforeDate, afterDate, and includeUnfilteredResults.

includeAds stores ads. liveAiOverview stores the AI overview and waits briefly when the page is still generating one. webTabRankingCheck compares the web-tab organic list. saveHtml puts the HTML on the item. saveHtmlToKeyValueStore stores a snapshot. All of these are off by default and use browser mode. maxConcurrency caps parallel pages.

No proxy or browser settings are needed. In browser mode, pages load in a real browser on a residential exit in countryCode, and the run's own residential proxy resolves Google's redirect links.

📤 Output

Each item has searchQuery (term, url, device, page, type, domain, countryCode, languageCode, locationUule, resultsPerPage) and the page url.

When the results page contains them: organicResults, paidResults (with Include ads), peopleAlsoAsk, relatedQueries, aiOverview, resultsTotal, and hasNextPage. An organic or paid row can include title, url, displayedUrl, description, type, position or adPosition, websiteTitle, date, averageRating, numberOfReviews, emphasizedKeywords, siteLinks, and on ads directUrl. Organic position counts on across a query's pages: page 2 starts after page 1's last result. A People Also Ask row can include question, title, url, answer, and date. A related query has title and url.

html is present only when saveHtml is on. htmlSnapshotUrl is present only when a snapshot was stored. A page that was not a results page is not saved; it is listed in the status message and in the OUTPUT record as failedInputs: [{input, reason}].