How to Scrape Twitter (X): Tweets, Likes, Reposts and Replies

To scrape Twitter (now X), give a scraper a public handle, a tweet URL or a search term, and save each post it returns: text, timestamp, likes, reposts, replies, bookmarks, media and the author's public profile. The fastest way is a ready-made scraper that runs in the cloud, from a web form or one API call, so you don't need X API credits or a headless browser of your own.

This guide shows both routes: no code, and code (curl, Python, Node.js and AI agents over MCP). Everything below comes from a real run on 22 October 2026: the exact input, the output, the log and what it cost. We also put the run next to the X API's own per-post price, because that's the comparison most people are making.

What data you can get from Twitter (X)

A public X profile, post or search shows posts and the counts X attaches to them. The Tweet Scraper on Scrape.ly saves one dataset row per post with these fields:

Keys appear only when X sent a value, so the scraper never invents a count. Profile timelines often leave out viewCount: none of the ten posts in our run had it. Private accounts and anything behind a login are not available.

How to scrape tweets without code

1. Open the Tweet Scraper in the Scrape.ly Store.
2. Pick a ready-made input. We used the preset Latest tweets from NASA and lowered it from 15 to 10 posts. "Try this input" opens it in the console.
3. Swap in the handle you want. This is the exact input of our run:

{
  "customMapFunction": "(object) => { return {...object} }",
  "maxItems": 10,
  "sort": "Latest",
  "tweetLanguage": "en",
  "twitterHandles": ["NASA"]
}

4. Start the run. Posts appear in the dataset as they are read.
5. Export the dataset as JSON, CSV or Excel.

The fields that matter most:

• twitterHandles: profiles to read, with or without @. The scraper walks each public timeline page by page until it reaches maxItems or the end.
• startUrls: profile, tweet, list or search URLs. A tweet URL returns that post plus the replies shown on its logged-out page.
• searchTerms: keywords or advanced-search text. The Top tweets about Artemis moon preset is a good starting point.
• maxItems: how many posts to save. You only pay for saved posts.
• start and end, onlyImage, onlyVideo, minimumFavorites: date window and filters for a profile timeline.

How to scrape Twitter with the API

Create an API token in the dashboard under Settings → API & Integrations and export it as SCRAPELY_TOKEN.

Our run finished in 6.5 seconds, so the synchronous endpoint works: it starts the run, waits up to 45 seconds and returns the posts as a JSON array. Add ?format=csv for CSV.

curl -X POST "https://api.scrape.ly/v2/actors/echofield~tweet-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"twitterHandles": ["NASA"], "maxItems": 10, "sort": "Latest", "tweetLanguage": "en"}'

For several handles or a few hundred posts, a run can pass 45 seconds. The call then answers 408 while the run keeps going. Don't call it again (that starts a second run); use a client, which waits as long as it takes.

Python (pip install scrapely-client-python), keeping only the account's own posts:

import os
from scrapely_client import ScrapelyClient

client = ScrapelyClient(os.environ["SCRAPELY_TOKEN"], api_url="https://api.scrape.ly")

run = client.actor("echofield/tweet-scraper").call(
    run_input={"twitterHandles": ["NASA"], "maxItems": 10, "sort": "Latest", "tweetLanguage": "en"}
)
print(run.status)  # SUCCEEDED, FAILED, ABORTED or TIMED-OUT
for post in client.dataset(run.default_dataset_id).iterate_items():
    if post["author"]["userName"] == "NASA" and not post.get("isRetweet"):
        print(post["createdAt"], post["likeCount"], post["text"][:60])

Node.js (npm install scrapely-client):

import { ScrapelyClient } from 'scrapely-client';

const client = new ScrapelyClient({ token: process.env.SCRAPELY_TOKEN, baseUrl: 'https://api.scrape.ly' });

const run = await client.actor('echofield/tweet-scraper').call({
  twitterHandles: ['NASA'],
  maxItems: 10,
  sort: 'Latest',
  tweetLanguage: 'en',
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.map((post) => [post.createdAt, post.likeCount, post.retweetCount]));

To let an AI agent do it, add the Scrape.ly MCP server. In Cursor, put this in ~/.cursor/mcp.json:

{
  "mcpServers": {
    "scrapely": {
      "url": "https://api.scrape.ly/mcp",
      "headers": { "Authorization": "Bearer YOUR_TOKEN" }
    }
  }
}

In Claude Code, run claude mcp add --transport http scrapely https://api.scrape.ly/mcp and sign in with /mcp. Then ask: "Use scrapely to run echofield/tweet-scraper on @NASA for 10 posts and give me a table of date, likes and reposts." The agent finds the Actor, reads its input schema, runs it and reads the results. See the MCP docs and the Python client docs.

A real run: Latest tweets from NASA

We ran the NASA preset with a 10-post limit on 22 October 2026. It returned 10 posts in 6.5 seconds and was charged $0.0034. Here are NASA's first five original posts from the run:

NASA is a verified government account that we named in the input, so its posts are shown as published. Other accounts a post mentions appear as [account 1], [account 2], because they weren't the target of the run.

The run shows two things worth knowing before you build on the data. First, a timeline mixes in reposts: two of the ten rows were NASA reposts, and two more were the reposted originals, written by two other verified NASA accounts. Filter on author.userName and isRetweet (as in the Python example) when you want only the account's own words. Second, a repost row carries a like count of 0, because likes belong to the original post.

The full record for the first post also had the media URL, the alt-text field, the thread ID and NASA's profile at the time of the run: 92,311,592 followers, 115 accounts followed, joined in December 2007.

The log, without the platform's start-up lines:

2026-10-22T18:25:38.470Z [Scrapely] Container started (setup 0.9 s)
2026-10-22T18:25:39.617Z [scrapely.actor] INFO  [Scrapely] Initializing API storage for run ce20e7dff34d4cbba
2026-10-22T18:25:39.755Z [scrapely.proxy_configuration] INFO  Platform proxy access check succeeded.
2026-10-22T18:25:43.257Z [scrapely.actor] INFO  Pushed 10 posts and 0 failed inputs
2026-10-22T18:25:43.257Z [scrapely.actor] INFO  [Status message]: Saved 10 posts.
2026-10-22T18:25:44.156Z [Scrapely] Container state: status=exited exitCode=0 oomKilled=false

Three and a half seconds of scraping for ten posts: the scraper reads the timeline with plain HTTP requests and Chrome's TLS fingerprint instead of starting a browser.

What it costs, and how it compares to the X API

The Tweet Scraper charges $0.0003 per post saved to the dataset, which is $0.30 per 1,000 posts. A run also pays for the platform resources it uses.

For our run: 10 posts × $0.0003 = $0.003 in post fees. The total charge was $0.0034, because the run also used $0.0002 of compute and about $0.0002 of residential proxy traffic and dataset writes. A run for 1,000 posts costs $0.30 in post fees plus usage of the same kind.

The X API pricing page lists pay-per-usage credits, checked on 22 October 2026: $0.005 per post read and $0.010 per user read. The same 10 posts through the X API cost 10 × $0.005 = $0.05, and 1,000 posts cost $5.00. That is more than 16 times the Tweet Scraper's post fee. The X API's pay-per-usage plans are also capped at 3 million post reads per monthly billing cycle; beyond that, X points you to an Enterprise plan.

On Scrape.ly, the free plan includes $5 of usage every month with no credit card, which covers thousands of posts at this price. You can set a maximum charge per run, and the run stops cleanly when it reaches it.

Twitter scraping: what to watch out for

• Search lags live X by a few hours. Keyword search finds public posts through Google's index of x.com and then reads each post back from X. Latest looks at the past day first, then widens to a week and a month. Use a handle when you need the newest posts.
• Location operators don't work in search. near:, geocode: and place: are not supported on the search route. Filters like language, images, video and minimum counts still work on a from:handle timeline.
• Views are not always there. viewCount appears only when X's response includes it, and profile timelines often leave it out. Plan your metrics around likes, reposts, replies and bookmarks.
• Blocking and rate limits. X rate-limits busy IP addresses. The scraper uses a residential proxy session and retries a blocked call on a new session, so you don't manage proxies yourself.
• One bad input doesn't end the run. An unsupported URL or a closed account is listed under failedInputs in the run's OUTPUT record, and the other handles still land in the dataset. Failed inputs cost nothing in post fees.
• Personal data. Posts by private people are personal data under the GDPR and the CCPA. Collect what you need, keep it secure, and don't build profiles of individuals without a lawful basis.

Is it legal to scrape Twitter (X)?

Public posts are visible to anyone without an account, and the Tweet Scraper reads only that public, logged-out view. Scrape.ly Store Actors collect only publicly available data that any visitor can see. That doesn't settle the question on its own. X's Terms of Service (effective 9 October 2026) say that "crawling or scraping the Services in any form, for any purpose without our prior written consent is expressly prohibited." How those terms bind you depends on where you are, whether you have an account, and what you do with the data.

Privacy law applies too: handles, names and post texts of private people are personal data. Use the data for research, monitoring or analysis, keep only what you need, and respect rate limits. This is not legal advice; if the data feeds a product, ask a lawyer about your case. Our guide to web scraping vs. using an API covers the trade-offs in more depth.

Why Scrape.ly: an X API alternative for public posts

• A fraction of the per-post price. $0.30 per 1,000 posts in event fees against $5.00 per 1,000 post reads on the X API's pay-per-usage plan.
• No developer account, no app review. Sign up, pick a preset, run. The same Actor works from the console, the API, a schedule or an AI agent.
• No monthly cap to plan around. You pay per saved post, and a maximum charge per run keeps the bill exactly where you set it.
• Agent-ready. The hosted MCP server lets Claude, Cursor or ChatGPT run the scraper and read the results in one conversation.

If you want a single endpoint for X data in your own product, see the Twitter (X) scraper API page.

FAQ

Can you scrape Twitter without the API?

Yes. The Tweet Scraper reads X's public, logged-out pages, so you need neither X API credits nor an X developer account. You run it from the Scrape.ly console or call it with your Scrape.ly token.

How do I scrape Twitter for free?

Scrape.ly's free plan includes $5 of usage every month with no credit card. At $0.0003 per post, our 10-post run cost under half a cent, so the free credit covers a lot of posts.

Can I scrape tweets by keyword or hashtag?

Yes. Put the keyword or hashtag in searchTerms and choose Top or Latest sorting. Search results come from Google's index of public x.com posts and are typically a few hours behind live X.

Can I scrape the replies to a tweet?

Put the tweet's status URL in startUrls. The scraper returns that post plus the replies included on its logged-out status page. Repliers are private people, so treat their handles as personal data.

Is there a Twitter scraper API?

Every Store Actor is an API: you start it with one HTTP call and read the dataset as JSON, CSV or Excel. The Twitter (X) scraper API page shows the endpoints in one place.