Top pages for a search as Markdown, 3 results

Input

The input this example runs RAG Web Browser with. “Try this input” opens it in the console, where you can run it as-is or change any field first.

Search term or URLquery

web scraping with python

Maximum resultsmaxResults

3

Google Search retriesserpMaxRetries

2

Select a scraping toolscrapingTool

Raw HTTP

Target page dynamic content timeoutdynamicContentWaitSecs

10

Remove cookie warningsremoveCookieWarnings

Yes

Remove HTML elements (CSS selector)removeElementsCssSelector

nav, footer, script, style, noscript, svg, img[src^='data:'], [role="alert"], [role="banner"], [role="dialog"], [role="alertdialog"], [role="region"][aria-label*="skip" i], [aria-modal="true"]

Target page max retriesmaxRequestRetries

1

Output formatsoutputFormats

markdown

Request timeoutrequestTimeoutSecs

240

HTML transformerhtmlTransformer

none

Desired browsing concurrencydesiredConcurrency

5

{
  "query": "web scraping with python",
  "maxResults": 3,
  "serpMaxRetries": 2,
  "scrapingTool": "raw-http",
  "dynamicContentWaitSecs": 10,
  "removeCookieWarnings": true,
  "removeElementsCssSelector": "nav, footer, script, style, noscript, svg, img[src^='data:'],\n[role=\"alert\"],\n[role=\"banner\"],\n[role=\"dialog\"],\n[role=\"alertdialog\"],\n[role=\"region\"][aria-label*=\"skip\" i],\n[aria-modal=\"true\"]",
  "maxRequestRetries": 1,
  "outputFormats": [
    "markdown"
  ],
  "requestTimeoutSecs": 240,
  "htmlTransformer": "none",
  "desiredConcurrency": 5
}