Social Media Scraping: Platforms, Data and the Rules

Social media scraping is the automated collection of public posts, videos, comments and engagement counts from platforms like Instagram, TikTok, X, YouTube, Reddit, LinkedIn and Facebook. Brands use social media scraping to measure their own and their competitors' content, researchers to study what people discuss, and agencies to find what works in a niche before they spend a budget on it.

This guide covers what each platform gives you and what it costs per 1,000 results, shows real runs on five platforms from October 2026, builds one Python script that puts them all in the same table, and explains the rules: privacy law, platform terms, and the line between public and logged-in data.

What social media scraping collects, platform by platform

Each platform has its own ready-made Actor on Scrape.ly. All of them read public pages without logging in and charge a fixed price per result.

Comments usually have their own, cheaper events (TikTok comments are $0.80 per 1,000), and the Store has dedicated Actors for Instagram profiles, Facebook pages, Threads and Pinterest when you need a narrower job.

What teams use it for

• Brand and competitor benchmarking. Pull every post from your account and three rivals', then compare posting frequency, formats and engagement.
• Content research. See which videos in a niche get millions of plays, what their captions and lengths look like, and which hashtags they use.
• Social listening. Collect posts and comments that mention your product, category or campaign, then score sentiment.
• Influencer and partner research. Shortlist accounts by audience size and engagement before you contact them.
• Trend tracking. Watch subreddits, hashtags or search terms daily and catch topics while they're rising.
• Research. Build datasets of public discourse for academic or policy work, with the privacy safeguards below.

Real data from five platforms

We ran the Actors between 22 and 23 October 2026 with small, public inputs: the reels of @natgeo on Instagram, a TikTok search for "pasta recipe", the @NASAJPL YouTube channel, the latest posts from @NASA on X, and the newest posts in r/Python.

Creators found by a search are private persons, so we show them as pseudonyms like [creator 1]; accounts we targeted on purpose, all verified organizations, keep their names. A few things this table teaches:

• Counts don't mean the same thing everywhere. Instagram reports plays, TikTok plays, YouTube views; X and Reddit expose no view count in these runs at all. Compare platforms on likes and comments, or compare rates within one platform.
• Rates make posts comparable. Likes per play were 1.8% for the top Instagram reel, 3.9% for the top TikTok and 1.6% for the top YouTube video. A rate tells you more than a raw count when the audiences differ in size.
• Searches and accounts answer different questions. The TikTok search surfaced the most-watched videos for a topic, from creators of any size. The account runs show one publisher's recent output. Pick the input that matches your question.
• Speed varies by platform. Most runs took under 13 seconds; Instagram reels took 72 seconds, because each reel needs several requests. Plan run timeouts accordingly.

Our platform guides go deeper into each run: Instagram, TikTok, X (Twitter), YouTube and Reddit.

A cross-platform workflow in Python

Every platform names its fields differently: Instagram's likesCount is TikTok's diggCount and X's likeCount. The script below runs five Actors in parallel, maps each to the same seven columns and writes one CSV. Create an API token under Settings → API & Integrations, export it as SCRAPELY_TOKEN, and pip install scrapely-client-python:

import csv
import os
from concurrent.futures import ThreadPoolExecutor
from scrapely_client import ScrapelyClient

client = ScrapelyClient(os.environ["SCRAPELY_TOKEN"], api_url="https://api.scrape.ly")

JOBS = {
    "instagram": ("echofield/instagram-scraper",
                  {"directUrls": ["https://www.instagram.com/natgeo/"], "resultsType": "reels", "resultsLimit": 5}),
    "tiktok": ("echofield/tiktok-scraper",
               {"searchQueries": ["pasta recipe"], "searchSection": "/video", "resultsPerPage": 5}),
    "youtube": ("echofield/youtube-scraper",
                {"startUrls": [{"url": "https://www.youtube.com/@NASAJPL"}], "maxResults": 5}),
    "x": ("echofield/tweet-scraper", {"twitterHandles": ["NASA"], "maxItems": 10, "sort": "Latest"}),
    "reddit": ("echofield/reddit-posts-scraper", {"subredditUrls": ["r/python"], "maxPostsCount": 10, "searchSort": "new"}),
}

NORMALIZE = {
    "instagram": lambda p: (p.get("ownerUsername"), p.get("timestamp"), p.get("videoPlayCount"),
                            p.get("likesCount"), p.get("commentsCount"), p.get("url")),
    "tiktok": lambda p: (p["authorMeta"].get("name"), p.get("createTimeISO"), p.get("playCount"),
                         p.get("diggCount"), p.get("commentCount"), p.get("webVideoUrl")),
    "youtube": lambda p: (p.get("channelName"), p.get("date"), p.get("viewCount"),
                          p.get("likes"), p.get("commentsCount"), p.get("url")),
    "x": lambda p: (p["author"].get("userName"), p.get("createdAt"), None,
                    p.get("likeCount"), p.get("replyCount"), p.get("url")),
    "reddit": lambda p: (p.get("communityName"), p.get("createdAt"), None,
                         p.get("score"), p.get("commentsCount"), p.get("postUrl")),
}

def run(platform):
    actor, run_input = JOBS[platform]
    result = client.actor(actor).call(run_input=run_input)
    items = client.dataset(result.default_dataset_id).iterate_items()
    return [(platform, *NORMALIZE[platform](item)) for item in items]

with ThreadPoolExecutor(max_workers=5) as pool:
    rows = [row for batch in pool.map(run, JOBS) for row in batch]

with open("social.csv", "w", newline="") as f:
    writer = csv.writer(f)
    writer.writerow(["platform", "account", "posted", "views", "likes", "comments", "url"])
    writer.writerows(rows)

for platform in JOBS:
    mine = [r for r in rows if r[0] == platform]
    print(f"{platform:10} {len(mine):3} posts  {sum(r[4] or 0 for r in mine):>9,} likes  "
          f"{sum(r[5] or 0 for r in mine):>7,} comments")

We ran the mapping step on the saved datasets of the runs above: 35 rows, with 732,110 likes across the five TikTok videos, 144,478 across the five Instagram reels, 12,453 across ten NASA posts on X, 5,912 across five JPL videos and 503 across ten Reddit posts. If your inputs include searches, replace the account value of creators you didn't target with a pseudonym before you share the file.

AI agents can run the same comparison over MCP: add https://api.scrape.ly/mcp to Cursor or Claude Code and ask "Use scrapely to get the last 10 posts from @NASA on X and the last 5 videos on the NASAJPL YouTube channel, and compare likes and comments."

Track it weekly without code

1. Save one input per platform. On each Actor's page, set the accounts, hashtags or searches you track and use Save as new… in the Input menu.
2. Create one automation. In Automations, click New automation, set When to On a schedule with Weekly, and add the saved inputs under Run (up to 10 Actors, all starting at the same time).
3. Keep only new posts. Under Keep, choose Only new items since the last run with the key fields url, webVideoUrl, postUrl, which cover the link field of each platform above.
4. Summarize. Turn on the AI step in Summarize mode: "The 5 posts with the most likes this week across all platforms, with one line on what they have in common." It's billed per use, like the AI chat.
5. Send. Post the summary to Slack and append the rows to Google Sheets to build a history. The automations guide lists every option.

Set Max cost per run on each run, and the run stops when it reaches it.

What it costs

Every Actor charges a fixed fee per result, plus platform usage. From the runs above:

• Instagram: 5 reels × $0.002 = $0.01 in result fees; $0.0189 charged in total (most of the rest was residential proxy traffic).
• TikTok: 5 videos × $0.0035 = $0.0175; $0.0179 in total.
• YouTube: 5 videos × $0.003 = $0.015; $0.0185 in total.
• X: 10 posts × $0.0003 = $0.003; $0.0034 in total.
• Reddit: 10 posts × $0.0014 = $0.014; $0.0142 in total.

A worked example: a weekly benchmark of 5 brands with 20 new posts per platform each, on Instagram, TikTok, YouTube and X, is 100 posts per platform per week, or 100 × ($0.002 + $0.0035 + $0.003 + $0.0003) = $0.88 a week in result fees, about $3.52 for four weeks. That fits inside the Free plan's $5 of monthly credit, before platform usage; the Starter plan at $19 a month includes $19 of credit.

Is social media scraping legal? The rules that matter

The Actors above read only public pages that any logged-out visitor can see. Three sets of rules still apply, and none of this is legal advice.

Privacy law applies to public posts. In their joint statement on data scraping, 16 data protection authorities said that "personal information that is publicly accessible is subject to data protection and privacy laws in most jurisdictions" (Office of the Privacy Commissioner of Canada, October 2024). Under GDPR that means a lawful basis, collecting only what you need, and keeping it no longer than necessary. Aggregate statistics about posts are low risk; profiles of individual people are not. The EU AI Act goes further for one use: it bans untargeted scraping of facial images to build facial recognition databases.

Logged-out is not the same as logged-in. In Meta v. Bright Data, a federal court in California held in January 2024 that Meta's terms "do not apply to Bright Data's logged-off scraping of publicly viewable data" (order). The reasoning turned on not being logged in. Once you sign in, you've accepted the platform's terms, which is why Scrape.ly's Store Actors never use accounts.

Platform terms and rate limits. Platforms restrict automated collection in their terms in different ways, and how those terms bind you depends on your jurisdiction and use. Keep request rates reasonable and read the terms for any commercial use.

Our practical rules: collect public content only, never private accounts or messages; pseudonymize people you didn't target, as our tables do; store counts and text, not profile pictures; and delete raw data once you've computed what you need. The full legal background, with the cases, is in our guide on whether web scraping is legal.

What to watch out for

• Media links expire. Instagram and TikTok serve images and videos from signed links that stop working, often within hours. Download the media you need during the run (the TikTok Scraper has a video download option at $0.80 per 1,000) instead of storing links.
• Counts are snapshots. A post's likes keep growing for days. Store the scrape date with every row and compare posts of similar age.
• Login-only content is out of scope. Private accounts, close-friends stories and messages aren't public, and the Actors don't try to reach them.
• Rate limits and blocking. Platforms throttle bursts from one address. The Actors run on Scrape.ly's residential and datacenter proxies inside the run, so you never manage IPs.
• Personal data in free text. Captions and comments can contain names, handles and phone numbers. Scrub them before sharing datasets.

Why Scrape.ly

• Every major platform, one account. Instagram, TikTok, X, YouTube, Reddit, LinkedIn and Facebook Actors, from one publisher, with one bill and one API.
• Fractions of a cent per post. From $0.30 per 1,000 tweets to $4.50 per 1,000 Facebook posts, with a maximum charge per run and a prepaid balance.
• Public data only. No accounts, no cookies, no logins to manage or lose.
• Monitoring built in. Weekly schedules, "only new items", AI summaries and deliveries to Slack and Google Sheets, plus MCP for agents.

FAQ

What is a social media scraper?

A social media scraper is a program that reads public pages on a platform and saves posts, profiles, comments and counts as structured data. On Scrape.ly each platform has its own Actor that you run from the console, the API or a schedule.

Is there a social media scraper API?

Yes. Every Actor runs through the same REST API: start a run with your input and read the results as JSON or CSV, or call the synchronous endpoint for short runs. Platform pages such as the Instagram API page show the request for each one.

Can you scrape Instagram without logging in?

Yes, for public profiles, posts, reels and hashtags. The Instagram Scraper reads what a logged-out visitor sees; private accounts stay private. Our Instagram scraping guide has a full run.

Which social media platform is cheapest to scrape?

X, at $0.30 per 1,000 tweets with the Tweet Scraper. Instagram costs $2.00 per 1,000 results and TikTok $3.50, plus platform usage, which is mostly proxy traffic on Instagram.

Is it legal to scrape Facebook?

Public Facebook pages can be scraped logged-out, and a US court found Meta's terms didn't cover logged-off scraping of publicly viewable data. Privacy law still applies to anything about people, so collect page-level data and aggregate where you can.

How do I scrape TikTok?

Enter search terms, hashtags, profiles or video URLs in the TikTok Scraper and set the number of results. See our TikTok scraping guide for inputs, a real run and costs.