Blog posts from a sitemap, 15 pages

Input

The input this example runs Structured Data Extractor – Schema.org, JSON-LD, OpenGraph & Meta with. “Try this input” opens it in the console, where you can run it as-is or change any field first.

Sitemap URLssitemapUrls

https://crawlee.dev

Max discovered pagesmaxCrawlPages

15

Link crawl depthcrawlDepth

0

Only these schema.org typesentityTypes

BlogPosting

Only pages with structured dataonlyPagesWithStructuredData

No

Output rowsoutputMode

One row per entity

Data to extractsyntaxes

json-ld, microdata, rdfa, opengraph, twitter, meta

Validation hintsvalidateSchema

Yes

Include raw markup dataincludeRawData

Yes

Include headings outlineincludeHeadings

Yes

Max parallel requestsmaxConcurrency

16

Page timeout (seconds)requestTimeoutSecs

20

{
  "sitemapUrls": [
    "https://crawlee.dev"
  ],
  "maxCrawlPages": 15,
  "outputMode": "entities",
  "entityTypes": [
    "BlogPosting"
  ],
  "crawlDepth": 0,
  "onlyPagesWithStructuredData": false,
  "syntaxes": [
    "json-ld",
    "microdata",
    "rdfa",
    "opengraph",
    "twitter",
    "meta"
  ],
  "validateSchema": true,
  "includeRawData": true,
  "includeHeadings": true,
  "maxConcurrency": 16,
  "requestTimeoutSecs": 20
}