--- title: "How to Scrape Twitter (X) Without the X API | Scrape.ly" description: "How to scrape Twitter (X) posts with likes, reposts and replies: no-code steps, a real NASA run with its cost, and code for curl, Python, Node and MCP." url: https://scrape.ly/blog/scrape-twitter markdown: https://scrape.ly/blog/scrape-twitter.md --- [Tutorials](https://scrape.ly/blog/category/tutorials.md) # How to Scrape Twitter (X): Tweets, Likes, Reposts and Replies Scrape Twitter (X) posts from any public profile, tweet URL or search, with no code or one API call. A real run on NASA's account, its output, its log and its exact cost, plus how it compares to the X API's per-post price. [Echofield Actor publisher on the Scrape.ly Store](https://scrape.ly/blog/author/echofield.md) - Published Oct 23, 2026 - Reading time 9 min read ![Pixel-art robot with an orange >\_ on its terminal head pulls a post card out of a phone whose feed is topped by the white X logo and drops it into a crate](https://scrape.ly/media/scrape-twitter.png?w=1080) To scrape Twitter (now X), give a scraper a public handle, a tweet URL or a search term, and save each post it returns: text, timestamp, likes, reposts, replies, bookmarks, media and the author's public profile. The fastest way is a ready-made scraper that runs in the cloud, from a web form or one API call, so you don't need X API credits or a headless browser of your own. This guide shows both routes: no code, and code (curl, Python, Node.js and AI agents over MCP). Everything below comes from a real run on 22 October 2026: the exact input, the output, the log and what it cost. We also put the run next to the X API's own per-post price, because that's the comparison most people are making. ## What data you can get from Twitter (X) A public X profile, post or search shows posts and the counts X attaches to them. The [Tweet Scraper](https://scrape.ly/store/echofield/tweet-scraper.md) on Scrape.ly saves one dataset row per post with these fields: | Field | What it holds | | ------------------------------------------------------------------------ | ----------------------------------------------------------------------------------- | | `id`, `url`, `twitterUrl` | The post ID and its x.com and twitter.com links | | `text`, `fullText` | The post text; `fullText` is the expanded text of a long post | | `createdAt`, `lang` | When it was posted (UTC) and its language | | `likeCount`, `retweetCount`, `replyCount`, `quoteCount`, `bookmarkCount` | Engagement counts, including real zeros | | `viewCount` | Views, when the response includes them | | `author` | Handle, display name, verified type, followers, following, bio, location, join date | | `media` | Photos and videos with type, URL, size and alt text | | `mentions`, `hashtags`, `urls`, `card` | Accounts, tags, links and link cards in the post | | `isReply`, `inReplyToId`, `inReplyToUserName` | Reply details | | `isRetweet`, `retweet` | Repost flag and the reposted post | | `isQuote`, `quote` | Quote flag and the quoted post | | `conversationId`, `source` | Thread ID and the app it was posted from | Keys appear only when X sent a value, so the scraper never invents a count. Profile timelines often leave out `viewCount`: none of the ten posts in our run had it. Private accounts and anything behind a login are not available. ## How to scrape tweets without code 1. Open the [Tweet Scraper](https://scrape.ly/store/echofield/tweet-scraper.md) in the Scrape.ly Store. 2. Pick a ready-made input. We used the preset [Latest tweets from NASA](https://scrape.ly/store/echofield/tweet-scraper/presets/twitter-handles.md) and lowered it from 15 to 10 posts. "Try this input" opens it in the console. 3. Swap in the handle you want. This is the exact input of our run: ```json { "customMapFunction": "(object) => { return {...object} }", "maxItems": 10, "sort": "Latest", "tweetLanguage": "en", "twitterHandles": ["NASA"] } ``` 4. Start the run. Posts appear in the dataset as they are read. 5. Export the dataset as JSON, CSV or Excel. The fields that matter most: - `twitterHandles`: profiles to read, with or without `@`. The scraper walks each public timeline page by page until it reaches `maxItems` or the end. - `startUrls`: profile, tweet, list or search URLs. A tweet URL returns that post plus the replies shown on its logged-out page. - `searchTerms`: keywords or advanced-search text. The [Top tweets about Artemis moon](https://scrape.ly/store/echofield/tweet-scraper/presets/search-terms-top.md) preset is a good starting point. - `maxItems`: how many posts to save. You only pay for saved posts. - `start` and `end`, `onlyImage`, `onlyVideo`, `minimumFavorites`: date window and filters for a profile timeline. Scrape.ly Store Tweet Scraper Public X (Twitter) posts from handles, tweet URLs or searches, with likes, reposts, replies, media and author details. $0.30 per 1,000 tweets. [Try it free](https://scrape.ly/store/echofield/tweet-scraper.md) [Browse the Store](https://scrape.ly/store.md) ## How to scrape Twitter with the API Create an API token in the dashboard under **Settings → API & Integrations** and export it as `SCRAPELY_TOKEN`. Our run finished in 6.5 seconds, so the synchronous endpoint works: it starts the run, waits up to 45 seconds and returns the posts as a JSON array. Add `?format=csv` for CSV. ```shell curl -X POST "https://api.scrape.ly/v2/actors/echofield~tweet-scraper/run-sync-get-dataset-items" -H "Authorization: Bearer $SCRAPELY_TOKEN" -H "Content-Type: application/json" -d '{"twitterHandles": ["NASA"], "maxItems": 10, "sort": "Latest", "tweetLanguage": "en"}' ``` For several handles or a few hundred posts, a run can pass 45 seconds. The call then answers `408` while the run keeps going. Don't call it again (that starts a second run); use a client, which waits as long as it takes. Python (`pip install scrapely-client-python`), keeping only the account's own posts: ```python import os from scrapely_client import ScrapelyClient client = ScrapelyClient(os.environ["SCRAPELY_TOKEN"], api_url="https://api.scrape.ly") run = client.actor("echofield/tweet-scraper").call( run_input={"twitterHandles": ["NASA"], "maxItems": 10, "sort": "Latest", "tweetLanguage": "en"} ) print(run.status) # SUCCEEDED, FAILED, ABORTED or TIMED-OUT for post in client.dataset(run.default_dataset_id).iterate_items(): if post["author"]["userName"] == "NASA" and not post.get("isRetweet"): print(post["createdAt"], post["likeCount"], post["text"][:60]) ``` Node.js (`npm install scrapely-client`): ```javascript import { ScrapelyClient } from 'scrapely-client'; const client = new ScrapelyClient({ token: process.env.SCRAPELY_TOKEN, baseUrl: 'https://api.scrape.ly' }); const run = await client.actor('echofield/tweet-scraper').call({ twitterHandles: ['NASA'], maxItems: 10, sort: 'Latest', tweetLanguage: 'en', }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items.map((post) => [post.createdAt, post.likeCount, post.retweetCount])); ``` To let an AI agent do it, add the Scrape.ly MCP server. In Cursor, put this in `~/.cursor/mcp.json`: ```json { "mcpServers": { "scrapely": { "url": "https://api.scrape.ly/mcp", "headers": { "Authorization": "Bearer YOUR_TOKEN" } } } } ``` In Claude Code, run `claude mcp add --transport http scrapely https://api.scrape.ly/mcp` and sign in with `/mcp`. Then ask: "Use scrapely to run echofield/tweet-scraper on @NASA for 10 posts and give me a table of date, likes and reposts." The agent finds the Actor, reads its input schema, runs it and reads the results. See the [MCP docs](https://scrape.ly/docs/mcp.md) and the [Python client docs](https://scrape.ly/docs/client-python.md). ## A real run: Latest tweets from NASA We ran the NASA preset with a 10-post limit on 22 October 2026. It returned 10 posts in 6.5 seconds and was charged $0.0034. Here are NASA's first five original posts from the run: | Posted (UTC) | Post starts with | Likes | Reposts | Replies | Bookmarks | Media | | ------------- | -------------------------------------------------------------------------------- | ----- | ------- | ------- | --------- | ----- | | Oct 21, 17:30 | "I just hear over and over again: 'I felt like I went on the mission with you'." | 1,567 | 208 | 147 | 74 | photo | | Oct 20, 18:00 | "LIVE: Get a preview of two upcoming spacewalks set for Oct. 29 and Nov. 5." | 1,146 | 174 | 122 | 59 | none | | Oct 20, 15:05 | "There's a lot of ground to cover when you're building a Moon Base." | 3,792 | 565 | 253 | 198 | video | | Oct 20, 15:05 | "@\[account 2] The journey is just beginning." | 451 | 73 | 14 | 29 | none | | Oct 20, 14:34 | "Welcome to the Artemis Accords, Uzbekistan" | 1,885 | 209 | 116 | 71 | photo | NASA is a verified government account that we named in the input, so its posts are shown as published. Other accounts a post mentions appear as `[account 1]`, `[account 2]`, because they weren't the target of the run. The run shows two things worth knowing before you build on the data. First, a timeline mixes in reposts: two of the ten rows were NASA reposts, and two more were the reposted originals, written by two other verified NASA accounts. Filter on `author.userName` and `isRetweet` (as in the Python example) when you want only the account's own words. Second, a repost row carries a like count of 0, because likes belong to the original post. The full record for the first post also had the media URL, the alt-text field, the thread ID and NASA's profile at the time of the run: 92,311,592 followers, 115 accounts followed, joined in December 2007. The log, without the platform's start-up lines: ```plaintext 20261022T18:25:38.470Z [Scrapely] Container started (setup 0.9 s) 20261022T18:25:39.617Z [scrapely.actor] INFO [Scrapely] Initializing API storage for run ce20e7dff34d4cbba 20261022T18:25:39.755Z [scrapely.proxy_configuration] INFO Platform proxy access check succeeded. 20261022T18:25:43.257Z [scrapely.actor] INFO Pushed 10 posts and 0 failed inputs 20261022T18:25:43.257Z [scrapely.actor] INFO [Status message]: Saved 10 posts. 20261022T18:25:44.156Z [Scrapely] Container state: status=exited exitCode=0 oomKilled=false ``` Three and a half seconds of scraping for ten posts: the scraper reads the timeline with plain HTTP requests and Chrome's TLS fingerprint instead of starting a browser. ## What it costs, and how it compares to the X API The Tweet Scraper charges $0.0003 per post saved to the dataset, which is $0.30 per 1,000 posts. A run also pays for the platform resources it uses. For our run: 10 posts × $0.0003 = $0.003 in post fees. The total charge was $0.0034, because the run also used $0.0002 of compute and about $0.0002 of residential proxy traffic and dataset writes. A run for 1,000 posts costs $0.30 in post fees plus usage of the same kind. The [X API pricing page](https://docs.x.com/x-api/getting-started/pricing) lists pay-per-usage credits, checked on 22 October 2026: **$0.005 per post read** and $0.010 per user read. The same 10 posts through the X API cost 10 × $0.005 = $0.05, and 1,000 posts cost $5.00. That is more than 16 times the Tweet Scraper's post fee. The X API's pay-per-usage plans are also capped at 3 million post reads per monthly billing cycle; beyond that, X points you to an Enterprise plan. On Scrape.ly, the free plan includes $5 of usage every month with no credit card, which covers thousands of posts at this price. You can set a maximum charge per run, and the run stops cleanly when it reaches it. ## Twitter scraping: what to watch out for - **Search lags live X by a few hours.** Keyword search finds public posts through Google's index of x.com and then reads each post back from X. `Latest` looks at the past day first, then widens to a week and a month. Use a handle when you need the newest posts. - **Location operators don't work in search.** `near:`, `geocode:` and `place:` are not supported on the search route. Filters like language, images, video and minimum counts still work on a `from:handle` timeline. - **Views are not always there.** `viewCount` appears only when X's response includes it, and profile timelines often leave it out. Plan your metrics around likes, reposts, replies and bookmarks. - **Blocking and rate limits.** X rate-limits busy IP addresses. The scraper uses a residential proxy session and retries a blocked call on a new session, so you don't manage proxies yourself. - **One bad input doesn't end the run.** An unsupported URL or a closed account is listed under `failedInputs` in the run's `OUTPUT` record, and the other handles still land in the dataset. Failed inputs cost nothing in post fees. - **Personal data.** Posts by private people are personal data under the GDPR and the CCPA. Collect what you need, keep it secure, and don't build profiles of individuals without a lawful basis. ## Is it legal to scrape Twitter (X)? Public posts are visible to anyone without an account, and the Tweet Scraper reads only that public, logged-out view. Scrape.ly Store Actors collect only publicly available data that any visitor can see. That doesn't settle the question on its own. X's [Terms of Service](https://x.com/en/tos) (effective 9 October 2026) say that "crawling or scraping the Services in any form, for any purpose without our prior written consent is expressly prohibited." How those terms bind you depends on where you are, whether you have an account, and what you do with the data. Privacy law applies too: handles, names and post texts of private people are personal data. Use the data for research, monitoring or analysis, keep only what you need, and respect rate limits. This is not legal advice; if the data feeds a product, ask a lawyer about your case. Our guide to [web scraping vs. using an API](https://scrape.ly/blog/web-scraping-vs-api.md) covers the trade-offs in more depth. ## Why Scrape.ly: an X API alternative for public posts - **A fraction of the per-post price.** $0.30 per 1,000 posts in event fees against $5.00 per 1,000 post reads on the X API's pay-per-usage plan. - **No developer account, no app review.** Sign up, pick a preset, run. The same Actor works from the console, the API, a schedule or an AI agent. - **No monthly cap to plan around.** You pay per saved post, and a maximum charge per run keeps the bill exactly where you set it. - **Agent-ready.** The hosted MCP server lets Claude, Cursor or ChatGPT run the scraper and read the results in one conversation. If you want a single endpoint for X data in your own product, see the [Twitter (X) scraper API](https://scrape.ly/api/twitter-x-api.md) page. ## FAQ ### Can you scrape Twitter without the API? Yes. The Tweet Scraper reads X's public, logged-out pages, so you need neither X API credits nor an X developer account. You run it from the Scrape.ly console or call it with your Scrape.ly token. ### How do I scrape Twitter for free? Scrape.ly's free plan includes $5 of usage every month with no credit card. At $0.0003 per post, our 10-post run cost under half a cent, so the free credit covers a lot of posts. ### Can I scrape tweets by keyword or hashtag? Yes. Put the keyword or hashtag in `searchTerms` and choose `Top` or `Latest` sorting. Search results come from Google's index of public x.com posts and are typically a few hours behind live X. ### Can I scrape the replies to a tweet? Put the tweet's status URL in `startUrls`. The scraper returns that post plus the replies included on its logged-out status page. Repliers are private people, so treat their handles as personal data. ### Is there a Twitter scraper API? Every Store Actor is an API: you start it with one HTTP call and read the dataset as JSON, CSV or Excel. The [Twitter (X) scraper API](https://scrape.ly/api/twitter-x-api.md) page shows the endpoints in one place. Scrape.ly Store Scrape Twitter (X) posts Run the NASA preset as-is, or swap in any public handle, tweet URL or search term. Pay $0.30 per 1,000 tweets, with a hard cap per run. [Try it free](https://scrape.ly/store/echofield/tweet-scraper.md) [Browse the Store](https://scrape.ly/store.md) About the author [Echofield](https://scrape.ly/blog/author/echofield.md) Actor publisher on the Scrape.ly Store Posts, reels, comments and creator stats, from TikTok and YouTube to Weibo and Bluesky. For social listening, creator research and catching a trend while it is still small. A Scrapely studio. - [Website](https://scrape.ly/users/echofield.md) Keep reading ## More in Tutorials - [![Pixel-art robot with an orange >\_ on its terminal head pulls an article page out of a newspaper vending box topped with the Google News logo and drops it into a crate](https://scrape.ly/media/how-to-scrape-news-articles.png?w=750)](https://scrape.ly/blog/how-to-scrape-news-articles.md) ### [How to Scrape News Articles: Headlines, Full Text and Dates](https://scrape.ly/blog/how-to-scrape-news-articles.md) [Web scraping news takes two steps: find the headlines for a topic, then extract each article's text, author and publish date. Both steps with ready-made scrapers, two real runs with their exact cost, Python code, and the copyright rules.](https://scrape.ly/blog/how-to-scrape-news-articles.md) - [![Pixel-art robot with an orange >\_ on its terminal head takes a job card with a briefcase icon off a cork job board topped with the blue Indeed logo and drops it into a crate](https://scrape.ly/media/scrape-indeed.png?w=750)](https://scrape.ly/blog/scrape-indeed.md) ### [How to Scrape Indeed Job Listings: Salaries, Companies and Apply Links](https://scrape.ly/blog/scrape-indeed.md) [Search Indeed by job title and city, or paste a company's jobs page, and get each listing as a row with salary, job type, company rating, full description and the employer's apply link. A real run, its cost, and code for curl, Python, Node.js and AI agents.](https://scrape.ly/blog/scrape-indeed.md) - [![Pixel-art robot with an orange >\_ on its terminal head pulls a chart slip with a rising line out of a monitor showing the Google Trends logo and a climbing line chart, and drops it into a crate](https://scrape.ly/media/scrape-google-trends.png?w=750)](https://scrape.ly/blog/scrape-google-trends.md) ### [How to Scrape Google Trends: Interest Over Time, Regions and Related Queries](https://scrape.ly/blog/scrape-google-trends.md) [Scrape Google Trends for any search term, country and time range, with no code or one API call, then plot the timeline in Python. A real coffee-vs-tea run with its output, log and exact cost, plus where pytrends and Google's own API stand today.](https://scrape.ly/blog/scrape-google-trends.md) ## The internet is full of data. Start owning yours. Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes. The advantage lasts forever. [Get Started Free](https://my.scrape.ly/register) [Chat with Support](mailto:support@scrape.ly?subject=Question%20about%20Scrape.ly\&body=Hi%2C%20I%20have%20a%20question%20about%20Scrape.ly.) No credit card. No setup calls.