Reddit Scraping: How to Collect Posts, Comments and Subreddit Data

Reddit scraping means collecting public posts, comment threads, community details and profiles from reddit.com into a structured file. The fastest way is a ready-made Actor: give it a subreddit, a keyword or a Reddit URL, run it, and export the results as JSON, CSV or Excel. You don't need a Reddit account, API key or cookies.

This guide shows the fields you get and how to scrape a subreddit with no code. It also covers curl, Python, JavaScript and AI agents, the cost per result and the limits. Because Reddit's terms are strict about scraping, it covers those too, along with the official Data API. It uses a real run from 22 October 2026: the 10 newest posts in r/Python.

What data you can get from Reddit

Every item is one flat JSON row. The dataType field tells you its shape: post, comment, community or user_profile. All timestamps are ISO 8601 in UTC.

What you don't get: moderator-only fields such as removedBy are always null for public scraping, and removedByCategory gives only the public removal reason. Private, quarantined or banned communities and suspended accounts are skipped, and user search and media tabs aren't supported.

How to scrape Reddit without code

1. Open the Reddit Posts Scraper in the Scrape.ly Store and sign in.
2. Pick the preset Newest posts in r/python, 10 posts. It fills in the input below.
3. Check the input. This is the exact JSON we ran:

{
  "subredditUrls": ["r/python"],
  "maxPostsCount": 10,
  "searchTerms": [],
  "onlyWithFlair": false,
  "includeNSFW": false,
  "searchPosts": true,
  "searchComments": false,
  "searchCommunities": false,
  "searchSort": "new",
  "searchTime": "all",
  "maxCommentsCount": 5,
  "maxCommunitiesCount": 2,
  "crawlCommentsPerPost": false,
  "maxCommentsPerPost": 5
}

4. Click Start. Our run finished in under 6 seconds.
5. Open the dataset and export it as JSON, CSV, Excel or HTML. The Posts & comments view shows the main columns. Communities and User profiles have their own views.

The input fields that matter most:

• subredditUrls: subreddit names or links (AskReddit, r/AskReddit or the full URL). It combines the new, top, hot, controversial and rising listings to reach far more posts than one subreddit page.
• startUrls: any public Reddit URL, such as a sorted subreddit page (/r/technology/top/?t=week), a post, a comment, a profile, a search page or r/popular.
• searchTerms: keywords. Each one runs its own Reddit search. searchPosts, searchComments and searchCommunities choose what to save, and withinCommunity keeps the search inside one subreddit.
• maxPostsCount: posts per keyword or URL, or the total for subredditUrls. The default is 20. Set it to 0 with a subreddit URL to get only the community profile and its rules.
• crawlCommentsPerPost and maxCommentsPerPost: also save each post's comment thread, with nested replies in thread order. You get more data, but the run is slower.

For a daily brand-monitoring run, set postedAfter to 1 day and schedule it.

How to scrape Reddit with Python and the API

Get your API token from Settings → API & Integrations in the dashboard and export it as SCRAPELY_TOKEN. The Actors API docs list every endpoint.

curl. This run took under 6 seconds, well inside the 45-second limit of the synchronous endpoint. The call returns the dataset items as a JSON array (add ?format=csv for CSV):

curl -X POST "https://api.scrape.ly/v2/actors/echofield~reddit-posts-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"subredditUrls": ["r/python"], "maxPostsCount": 10, "searchTerms": [], "onlyWithFlair": false, "includeNSFW": false, "searchPosts": true, "searchComments": false, "searchCommunities": false, "searchSort": "new", "searchTime": "all", "maxCommentsCount": 5, "maxCommunitiesCount": 2, "crawlCommentsPerPost": false, "maxCommentsPerPost": 5}'

Python. Install the client with pip install scrapely-client-python. call() waits for the run however long it takes, which you'll want once you turn on comment threads. See the Python client docs.

import os
from scrapely_client import ScrapelyClient

client = ScrapelyClient(os.environ["SCRAPELY_TOKEN"], api_url="https://api.scrape.ly")

run = client.actor("echofield/reddit-posts-scraper").call(
    run_input={
        "subredditUrls": ["r/python"], "maxPostsCount": 10, "searchTerms": [],
        "onlyWithFlair": False, "includeNSFW": False, "searchPosts": True,
        "searchComments": False, "searchCommunities": False, "searchSort": "new",
        "searchTime": "all", "maxCommentsCount": 5, "maxCommunitiesCount": 2,
        "crawlCommentsPerPost": False, "maxCommentsPerPost": 5,
    }
)
print(run.status)  # SUCCEEDED, FAILED, ABORTED or TIMED-OUT
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item["dataType"], item["score"], item["title"], item["postUrl"])

JavaScript. Install with npm install scrapely-client. See the JavaScript client docs.

import { ScrapelyClient } from 'scrapely-client';

const client = new ScrapelyClient({ token: process.env.SCRAPELY_TOKEN, baseUrl: 'https://api.scrape.ly' });

const run = await client.actor('echofield/reddit-posts-scraper').call({
  subredditUrls: ['r/python'], maxPostsCount: 10, searchTerms: [],
  onlyWithFlair: false, includeNSFW: false, searchPosts: true,
  searchComments: false, searchCommunities: false, searchSort: 'new',
  searchTime: 'all', maxCommentsCount: 5, maxCommunitiesCount: 2,
  crawlCommentsPerPost: false, maxCommentsPerPost: 5,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.length, items[0]);

AI agents (MCP). Scrape.ly runs a hosted MCP server at https://api.scrape.ly/mcp. In Cursor, add it to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "scrapely": {
      "url": "https://api.scrape.ly/mcp",
      "headers": { "Authorization": "Bearer YOUR_TOKEN" }
    }
  }
}

In Claude Code, run claude mcp add --transport http scrapely https://api.scrape.ly/mcp, then /mcp to sign in. The agent uses search-actors, fetch-actor-details, call-actor and get-actor-output. Try: "Use scrapely to run echofield/reddit-posts-scraper for the 10 newest posts in r/python and give me a table of title, flair, score and comment count." The MCP docs cover OAuth and other clients.

A real run: newest posts in r/Python

We ran the preset on 22 October 2026. It saved 10 posts in 5.849 seconds with 256 MB of memory, and the run was charged $0.0142. All rows came from r/Python. Five of them, with the community and URL columns left out for width:

The author names read [redditor 1] to [redditor 5] because we replaced the usernames for this article; your dataset holds the real Reddit username in authorName.

The console excerpt:

2026-10-22T15:23:09.681Z [scrapely.proxy_configuration] INFO  Platform proxy access check succeeded.
2026-10-22T15:23:09.681Z [scrapely.actor] INFO  Starting 1 targets (proxy DATACENTER).
2026-10-22T15:23:11.584Z [scrapely.actor] INFO  Saved 10 results (10 posts) from 1 targets. Requests: 2, retries: 0.
2026-10-22T15:23:11.786Z [scrapely.actor] INFO  [Status message]: Saved 10 results (10 posts) from 1 targets.
2026-10-22T15:23:12.231Z [Scrapely] Container state: status=exited exitCode=0 oomKilled=false

The five posts run from 11:44 back to just after midnight UTC on 22 October, newest first. Score and discussion don't always move together: "Python - Numpy Pandas" has a score of 0 but 11 comments. The run needed 2 requests on the default datacenter proxy, with no retries.

What it costs

The Actor charges one event: $0.0014 per result, where a result is one saved post, comment, community or user profile. There is no start fee. Platform usage (compute, proxy traffic and storage) is billed on top.

For this run, 10 posts × $0.0014 = $0.014 in result fees. The run was charged $0.0142 in total because platform usage added about $0.0001 in compute, $0.00005 in key-value store writes, $0.00003 in datacenter proxy traffic and $0.000005 in dataset writes.

At the same price, 1,000 results cost $1.40 in event fees. Comments count as results too. So 3 posts with crawlCommentsPerPost on and up to 20 comments each is up to 63 results, or about $0.088. Set a maximum total charge on a run to cap your spend.

Limits and common problems

• About 1,000 items per listing or search. That's Reddit's ceiling. subredditUrls combines several listings to go past it.
• Vote counts are fuzzed. Reddit rounds and fuzzes scores, on the website and in the data alike. Treat score as approximate.
• Comment search can be skipped. It relies on Reddit's public search page, which returns 10 comments per page. If Reddit challenges every attempt, those keywords are skipped and reported in the run summary, while posts and communities still come through.
• Skipped targets. Private, quarantined or banned communities and suspended accounts are skipped. The run summary (RUN-SUMMARY in the key-value store) lists each one with a reason such as private or banned.
• NSFW is off by default. Posts marked 18+ are dropped unless you set includeNSFW. Filters like this reduce the count you get.
• Proxy refusals. Reddit limits requests per IP address. The default datacenter proxy rotates exits, and after repeated refusals the Actor moves new sessions to residential proxies on its own.

Is it legal to scrape Reddit?

The Actor reads only pages anyone can open without logging in, but Reddit's terms restrict scraping clearly. Reddit's User Agreement says you may not collect data from the Services by automated means except as the terms or a separate agreement allow. It adds that "scraping the Services without Reddit's prior written consent is prohibited." The Data API Terms also bar using user content to train AI models without the rightsholders' permission. Usernames and posts can be personal data, so privacy laws such as the GDPR and CCPA may apply. Collect only what you need, drop usernames you don't use, and delete content its authors remove. This is general information, not legal advice. For research or commercial projects, get advice for your case.

Alternatives

• Reddit's official Data API. According to the Data API Wiki, you request access, authenticate with a registered OAuth token, and the free tier allows 100 queries per minute per OAuth client ID. Commercial use needs a separate agreement under the Data API Terms. It's the better choice for long-running products, apps that post or moderate, and anyone who needs a sanctioned route.
• PRAW. The Python Reddit API Wrapper, used in the CUNY and GeeksforGeeks tutorials. It makes the Data API easy from Python, but you still need API access first. Older tutorials skip the access request.
• Open-source scrapers. GitHub's reddit-scraper topic lists community projects. Check when each was last updated before you rely on it.
• Build it yourself. A headless browser or plain HTTP client works for small jobs. You handle per-IP rate limits, comment pagination ("load more" branches) and changes to Reddit's pages.

FAQ

Can you scrape Reddit without the API?

Technically, yes. The Actor reads public pages logged out, with no account, API key or cookies. But Reddit's User Agreement prohibits scraping without its prior written consent. If you need a route Reddit has approved, use the Data API or sign an agreement with Reddit.

How do I scrape a subreddit?

Put the subreddit's name in subredditUrls and set maxPostsCount to the total you want. For one sort order, such as this week's top posts, paste that listing's URL into startUrls instead.

Can I scrape Reddit comments?

Yes. Turn on crawlCommentsPerPost to save each post's thread, up to maxCommentsPerPost comments, with depth and parentId so you can rebuild the tree. To find comments by keyword, use searchTerms with searchComments on.

How much does Reddit scraping cost?

$0.0014 per saved post, comment, community or profile, so $1.40 per 1,000 results, plus platform usage. Our 10-post run was charged $0.0142 in total.