Substack Scraper

📰 Substack Scraper

Scrape any Substack newsletter, post, author profile or keyword search into a clean dataset: titles, full text of free posts, publish dates, reactions, restacks, comments with replies, paid/free status, subscriber counts and author details. No login, no browser, and fast: it reads Substack's own public JSON, so a newsletter's latest posts arrive in seconds.

✨ Why use Substack Scraper?

• ⚡ Fast — pure HTTP against Substack's public API, no page rendering. Full archives of thousands of posts are a single run.
• 🔗 Every way in — *.substack.com newsletters, custom domains (like lennysnewsletter.com), bare subdomain names, single post links, substack.com/@handle profiles, open.substack.com share links and keyword search.
• 📝 Full content — the complete HTML and a ready-to-use plain-text version of every free post, plus word count, cover image, tags and section.
• 💬 Comment threads — optional comments with nested replies, likes, dates and an "is the author" flag.
• 📊 Newsletter data on every row — subscriber count, subscriber range, paid-subscriber range, description, logo, language and the main author.
• 💸 Cheap and honest — you pay per post you get; paid posts are never reported as full content.

🎯 Use cases

• 📈 Newsletter benchmarking — compare posting cadence, reactions, restacks and comment counts across competing newsletters.
• 🤝 Sponsorship and partnership research — find newsletters on a topic and size them by subscriber count before you reach out.
• 🧠 Content and AI datasets — collect long-form articles with clean plain text for analysis, search or training (respecting authors' rights).
• 📰 Media monitoring — track what writers publish about a keyword, brand or industry, on a schedule.
• 💬 Audience research — read the comment threads under popular posts to see what readers care about.
• 🗂️ Archive backups — export your own newsletter's full public archive to JSON, CSV or Excel.

🚀 How to use

1. Paste one or more newsletter, post or profile URLs, or add search queries.
2. Set Max posts per newsletter (default 10, 0 = the whole archive) and Max posts per search query.
3. Optionally add filters (date range, post type, free/paid, minimum reactions) and turn on Include comments.
4. Click Start.
5. Download the results as JSON, CSV, Excel or HTML, or read them through the API.

📥 Input

{
  "startUrls": ["https://www.lennysnewsletter.com", "https://www.noahpinion.blog"],
  "searchQueries": ["climate policy"],
  "maxPostsPerNewsletter": 10,
  "maxSearchResults": 10,
  "startDate": "2026-01-01",
  "includeComments": true,
  "maxCommentsPerPost": 20
}

📤 Output

One row per post.

Sample row (shortened; commenter names are fictional):

{
  "type": "post",
  "postId": 220058014,
  "title": "A Nobel for New Keynesian macro, at last!",
  "subtitle": "It’s about time.",
  "url": "https://www.noahpinion.blog/p/a-nobel-for-new-keynesian-macro-at",
  "publishedAt": "2026-10-13T05:28:43.757Z",
  "postType": "newsletter",
  "audience": "everyone",
  "isPaid": false,
  "wordcount": 2753,
  "reactionCount": 389,
  "commentCount": 19,
  "restacks": 46,
  "bodyText": "I have a Google Doc with a long list of posts I’ve been meaning to write…",
  "hasFullContent": true,
  "authorName": "Noah Smith",
  "authorHandle": "noahpinion",
  "publicationName": "Noahpinion",
  "publicationUrl": "https://www.noahpinion.blog",
  "subscriberCount": "460,000",
  "subscriberRange": "Hundreds of thousands of subscribers",
  "paidSubscriberRange": "Tens of thousands of paid subscribers",
  "comments": [
    {
      "id": 359697984,
      "body": "Great summary of why this prize matters.",
      "date": "2026-10-13T05:47:38.273Z",
      "name": "Jane Reader",
      "handle": "janereader",
      "reactionCount": 20,
      "isAuthor": false,
      "replies": [
        { "id": 359700567, "body": "Thanks!", "name": "Noah Smith", "isAuthor": true, "replies": [] }
      ]
    }
  ],
  "commentsScraped": 2,
  "sourceUrl": "https://www.noahpinion.blog",
  "scrapedAt": "2026-10-14T21:40:12.31Z"
}

💰 Pricing

Pay per result, no monthly fee:

• Post with full content: $1.50 per 1,000 posts.
• Post metadata only (content turned off, or a paid post where only the preview is public): $0.80 per 1,000 posts.
• Comment (each comment or reply when comments are on): $0.40 per 1,000 comments.

Example: 1,000 free posts with full content ≈ $1.50; the same 1,000 posts as metadata only ≈ $0.80. Platform usage (a small amount of compute and datacenter proxy traffic) is billed separately and is usually a few cents per thousand posts.

❓ FAQ

Is it legal to scrape Substack? The actor reads only public data that Substack serves to any visitor without logging in. Respect authors' copyright and Substack's terms when you reuse the content, and handle personal data (like commenter names) according to the laws that apply to you.

Do I need a Substack account or a proxy? No account. Datacenter proxies are on by default. If Substack blocks a datacenter exit, the actor retries that request through a residential exit automatically, so a run doesn't lose results.

Can I get the text of paid posts? No. Paid posts return all metadata, engagement numbers, the teaser and the public preview Substack shows to non-subscribers, never the paywalled text.

How many posts can I get? Set maxPostsPerNewsletter to 0 for a newsletter's whole public archive. Search returns as many posts as Substack's search offers for a query (typically a few hundred).

Why is a field empty? Substack doesn't fill every field for every post: subscriber counts are only public when the writer shows them, podcast fields only exist on podcast episodes, and bodyHtml is empty for paid posts.

Can I schedule it or use the API? Yes. Schedule runs to track new posts, and read the dataset through the API or export it to JSON, CSV or Excel.

⚠️ Limits

• Paywalled text is never returned, only public previews.
• Substack Notes, chats and private (invite-only) newsletters are not covered.
• Keyword search results are ordered by Substack's relevance, not by date; date filters are applied to what search returns.
• Subscriber counts are Substack's public figures (exact count or a range like "Tens of thousands"), not private analytics.