Twitter Sentiment Analysis in Python: Score Real Tweets With VADER

Twitter sentiment analysis is the process of collecting posts from X (formerly Twitter) about a brand, product or topic and scoring each one as positive, negative or neutral, so you can see how people feel and when that changes. In Python, the shortest working version is two steps: pull the tweets with a scraper, then score their text with VADER, a sentiment model built for social media.

This tutorial does both with real data. On 26 October 2026 we collected the 20 most recent public posts mentioning "Delta Air Lines" with the Tweet Scraper, then scored them locally with VADER. The result is a useful lesson in itself: the most-liked post in the sample, a customer saying they will never fly the airline again, scored as perfectly neutral. You'll see why, and two tested fixes. To line sentiment up with share prices, see how to scrape Yahoo Finance.

Twitter sentiment analysis methods at a glance

VADER's README describes it as "a lexicon and rule-based sentiment analysis tool that is specifically attuned to sentiments expressed in social media". It returns four numbers per text: neg, neu and pos proportions, and a compound score from -1 (most negative) to +1 (most positive). The usual thresholds are compound ≥ 0.05 for positive and ≤ -0.05 for negative.

Step 1: Collect tweets about your brand

The Tweet Scraper reads public X posts from profiles, tweet URLs, conversation IDs and searches, without an X account or API key. For sentiment you usually want what other people say about a brand, so use a keyword search. This was our input:

{
  "searchTerms": ["Delta Air Lines"],
  "sort": "Latest",
  "maxItems": 20
}

Latest returns recent posts newest first. The Actor's README explains that search matches against a search index of public x.com posts, then reads each post back from X before storing it; it looks at the past day first and widens to the past week and month, and is typically a few hours behind live X. Top returns the most relevant posts instead. In the console, open the Actor, paste the input (or start from the preset Top tweets about Artemis moon and change the search term), and click Start.

From code, create an API token under Settings → API & Integrations and export it as SCRAPELY_TOKEN. Our run took 98 seconds, so start it asynchronously and wait:

# 1. start the run; the response's data.id is the run ID
curl -X POST "https://api.scrape.ly/v2/actors/echofield~tweet-scraper/runs" \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchTerms": ["Delta Air Lines"], "sort": "Latest", "maxItems": 20}'
# 2. wait (up to 60 s per call; repeat until status is SUCCEEDED)
curl "https://api.scrape.ly/v2/actor-runs/RUN_ID?waitForFinish=60" \
  -H "Authorization: Bearer $SCRAPELY_TOKEN"
# 3. read the tweets
curl "https://api.scrape.ly/v2/actor-runs/RUN_ID/dataset/items" \
  -H "Authorization: Bearer $SCRAPELY_TOKEN"

Or in Python (pip install scrapely-client-python), where call() waits for the run however long it takes:

import os
from scrapely_client import ScrapelyClient

client = ScrapelyClient(os.environ["SCRAPELY_TOKEN"], api_url="https://api.scrape.ly")

run = client.actor("echofield/tweet-scraper").call(
    run_input={"searchTerms": ["Delta Air Lines"], "sort": "Latest", "maxItems": 20}
)
tweets = [
    {"text": t.get("fullText") or t["text"], "lang": t.get("lang"),
     "likeCount": t.get("likeCount", 0), "retweetCount": t.get("retweetCount", 0)}
    for t in client.dataset(run.default_dataset_id).iterate_items()
]
print(run.status, len(tweets), "tweets")

Each item also carries createdAt, replyCount, quoteCount, bookmarkCount, often viewCount, the author's follower count and verification flags, media and mentions. fullText holds the whole text of long posts, where text is cut short.

What the run returned

The run saved 20 posts in 98 seconds. Its log shows the search widening from the past day to the past week to fill the 20:

2026-10-26T14:26:31.293Z [scrapely.actor] INFO  Recent posts window qdr:d stored 6 posts so far
2026-10-26T14:27:12.937Z [scrapely.actor] INFO  Recent posts window qdr:w stored 20 posts so far
2026-10-26T14:27:15.511Z [scrapely.actor] INFO  Pushed 20 posts and 0 failed inputs
2026-10-26T14:27:15.511Z [scrapely.actor] INFO  [Status message]: Saved 20 posts.

The 20 posts came from 15 accounts, posted between 23 and 26 October 2026: travel-news headlines (one news account posted 5 of them), aviation photos, stock-market lists, two posts in Turkish and Indonesian by X's language tag, and one viral complaint. Here are six, with the scores from the steps below. Accounts are pseudonymised ([account 12]) because they were found by a search, and handles inside the text are masked as @[user].

Step 2: Score each tweet with VADER

Install vaderSentiment (we used 3.3.2) and score every English tweet. This script reads the tweets from a JSON file, so you can run it on a saved sample; pass it the tweets list from Step 1 instead to go straight from the scraper:

import json
from collections import Counter

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

analyzer = SentimentIntensityAnalyzer()

def label(compound):
    if compound >= 0.05:
        return "positive"
    if compound <= -0.05:
        return "negative"
    return "neutral"

with open("tweets_sample.json") as f:
    tweets = json.load(f)

scored = []
for t in tweets:
    if t.get("lang") != "en":
        continue
    scores = analyzer.polarity_scores(t["text"])
    scored.append({**t, "compound": scores["compound"], "label": label(scores["compound"])})

counts = Counter(t["label"] for t in scored)
print(f"{len(scored)} English tweets:", dict(counts))
print("average compound:", round(sum(t["compound"] for t in scored) / len(scored), 3))
for t in sorted(scored, key=lambda t: t["compound"]):
    print(f'{t["compound"]:+.3f}  {t["label"]:<8}  {t["text"][:70]}')

Run on our 20 tweets (uv run --with vaderSentiment python sentiment.py), it kept the 18 tagged English and printed, in part:

18 English tweets: {'positive': 11, 'negative': 3, 'neutral': 4}
average compound: 0.265
-0.421  negative  Delta Air Lines Denies Passenger's $170,000 Food And Sticky Seat Claim
-0.296  negative  Delta Air Lines Passenger 'Yelled At' For Simple Request During Allerg
-0.153  negative  Delta Air Lines Flight Attendant Praised After Parent Abandons Infant'
+0.000  neutral   never flying @[user] airlines again and this time i mean it https://t.
+0.361  positive  Delta Air Lines Apologizes After Gate Agent Enforces Incorrect Bag Rul
+0.735  positive  Most Valuable Airlines by Market Cap  Delta Air Lines (USA): $61.0 Bil
+0.902  positive  Here is every stock currently in the Wedbush ReturnOnLeadership US Lar
+0.919  positive  Flying over Newfoundland Canada on United Airlines The JETS are packed

If you'd rather use NLTK, it ships the same model. nltk.download("vader_lexicon") once, then from nltk.sentiment import SentimentIntensityAnalyzer. On our sample it returned identical compound scores (-0.296 for the "Yelled At" headline, 0.0 for the complaint).

Where VADER goes wrong on real tweets

Taken at face value, the sample says sentiment about the airline is mildly positive: 11 positive, 3 negative, an average of +0.265. Read the rows and that falls apart:

• The strongest negative post scored zero. "never flying … again" contains no word in VADER's lexicon. The negativity is in the phrase and the intent, not in any single word. That post had 4,616 likes, 279 reposts and 85 replies, more than every other post in the sample combined.
• Headlines are not opinions. "Praised After Parent Abandons Infant's Mess" came out negative because of "abandons" and "mess". "Apologizes After Gate Agent Enforces Incorrect Bag Rule" came out positive. A news account summarising an incident isn't expressing sentiment about the brand at all.
• Lists and prices read as praise. The market-cap list (+0.735) and an ETF holdings list (+0.902) are neutral facts.
• Mentions aren't always about the brand. The +0.919 post praises a rival airline's service and mentions Delta only as a stock pick.

This is normal for lexicon models, and the fix is not to abandon them, but to give them better input and better aggregation.

Step 3: Weight by engagement and catch phrases

Three changes make the number honest: score only English posts, add a small rule for phrases the lexicon misses, and weight each tweet by how many people engaged with it, since one complaint with 4,616 likes says more than five headlines nobody liked. With pandas (we used 3.1.0):

import json
import re

import pandas as pd
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

analyzer = SentimentIntensityAnalyzer()
NEGATIVE_PHRASES = re.compile(r"\bnever (flying|fly|booking|using|buying)\b|\bnever again\b|\bworst\b", re.I)

def score(text):
    compound = analyzer.polarity_scores(text)["compound"]
    if NEGATIVE_PHRASES.search(text) and compound > -0.5:
        compound = -0.5
    return compound

rows = json.load(open("sample_rows.json"))
df = pd.DataFrame(rows)
df = df[df["lang"] == "en"].copy()
df["compound"] = df["text"].map(score)
df["label"] = pd.cut(df["compound"], [-1.01, -0.05, 0.05, 1.01], labels=["negative", "neutral", "positive"])
df["weight"] = 1 + df["likeCount"] + 2 * df["retweetCount"]

print(df["label"].value_counts().to_dict())
print("plain average:", round(df["compound"].mean(), 3))
print("engagement-weighted:", round((df["compound"] * df["weight"]).sum() / df["weight"].sum(), 3))

It printed:

{'positive': 11, 'negative': 4, 'neutral': 3}
plain average: 0.237
engagement-weighted: -0.471

Same 18 tweets, opposite conclusion. Weighted by engagement, the conversation about the airline that week was clearly negative, driven by one complaint that thousands of people agreed with. That's the number a social team should see. Two more filters help in production: drop or tag accounts that post headlines with links and hashtags (news and promo accounts), and track replies and quote posts separately, since that's where customers talk back.

A tweet sentiment analysis model trained on tweets

For better labels per tweet, use a transformer trained on Twitter data. The Cardiff NLP model twitter-roberta-base-sentiment-latest is, per its model card, "a RoBERTa-base model trained on ~124M tweets from January 2018 to December 2021, and finetuned for sentiment analysis with the TweetEval benchmark". We ran it on the same 18 tweets with transformers 5.20.2 and torch 2.14.1, after replacing handles with @user and links with http as the model card recommends:

from transformers import pipeline

clf = pipeline("sentiment-analysis", model="cardiffnlp/twitter-roberta-base-sentiment-latest")
texts = [" ".join("@user" if w.startswith("@") else "http" if w.startswith("http") else w
                  for w in t["text"].split()) for t in tweets if t["lang"] == "en"]
for t, res in zip(texts, clf(texts, truncation=True)):
    print(res["label"], round(res["score"], 2), t[:70])

It labelled 12 neutral, 3 positive and 3 negative, and it got the hard cases right: the "never flying" complaint negative with 0.92 confidence, the market-cap and ETF lists neutral, the "Flight Attendant Praised" headline positive. The model files are about 500 MB, and our first run took about four minutes including the download of PyTorch and the model; after that, everything is cached locally and runs on a laptop CPU. A practical split: VADER for a quick daily dashboard over everything, the transformer for the posts that matter (high engagement, replies to your account).

Run it every day without code

Sentiment is a trend, so schedule the collection. In Scrape.ly's Automations, create a new automation, set When to On a schedule (daily at 9:00 is the default), add the Tweet Scraper with your saved input, and choose Only new items since the last run with id as the key field, so each day's batch holds only new posts. Send the rows to Google Sheets, a webhook or an S3/R2 bucket, and point your scoring script at them. An optional AI step can summarise each batch ("What are people complaining about today?"); it's billed per use, like the AI chat.

AI agents can run the same collection over MCP: connect the hosted Scrape.ly MCP server and ask "Use scrapely to get the 50 latest tweets about Delta Air Lines with echofield/tweet-scraper and tell me the main complaints." Our Twitter MCP guide goes deeper.

What it costs

The Tweet Scraper charges $0.00038 per saved tweet ($0.38 per 1,000), with platform usage included. Our 20-tweet run cost 20 × $0.00038 = $0.0076 at list price; the account was billed $0.0061 because its plan has a Store discount. A daily brand check of 500 tweets is 500 × $0.00038 = $0.19 a day, about $5.70 for 30 days. VADER and the transformer run locally for free. The Free plan's $5 of monthly credit covers about 13,000 tweets.

What to watch out for

• Search is a few hours behind. The Latest search route is typically a few hours behind live X, so posts from the last hour or two may be missing. For an account's own posts, use twitterHandles instead, which reads the profile timeline.
• Language tags. X's lang field is a guess; one English-looking aviation post in our sample was tagged Indonesian. Filter on it, but spot-check what you drop.
• Sarcasm and context. No model reads sarcasm reliably. Review a sample by hand before you report a number, and keep the raw text next to each score.
• Small samples swing. One viral post flipped our average. Report counts and the weighted score together, and look at the trend over weeks rather than a single day.
• Personal data. Handles and post texts of private people are personal data. Aggregate, don't profile individuals, and keep only what you need.
• Terms. X's terms restrict scraping; our guide to scraping Twitter covers what they say and the legal context. This isn't legal advice.

Why Scrape.ly for Twitter sentiment analysis

• Tweets without the X API. Public posts from searches, profiles and tweet URLs for $0.38 per 1,000, no developer account, no app approval. The Twitter (X) scraper API page lists every field.
• Engagement on every row. Likes, reposts, replies, quotes, bookmarks and often views come with each tweet, which is what makes engagement weighting possible.
• Schedules and filters built in. Daily runs, "only new items", Google Sheets and webhooks without a server.
• The rest of social too. Reddit, YouTube comments, TikTok and Instagram have their own Actors; see social media scraping for a cross-platform workflow, and review monitoring for the same scoring idea on Google, Yelp and Trustpilot reviews.

FAQ

What is the best model for Twitter sentiment analysis in Python?

Start with VADER: it needs no training, runs on thousands of tweets in seconds and handles emoji, capitals and negation. For better accuracy on short, informal posts, use a transformer trained on tweets such as Cardiff NLP's twitter-roberta-base-sentiment-latest. In our test it caught a complaint that VADER scored as neutral.

How do I get tweets for sentiment analysis without the X API?

Use a scraper that reads public posts. The Tweet Scraper takes search terms, handles or tweet URLs and returns text, timestamps and engagement counts as JSON or CSV, for $0.38 per 1,000 tweets.

Is VADER good for tweet sentiment analysis?

It's a solid baseline built for social media text, and its scores are easy to explain. It misses sentiment carried by phrases ("never flying again"), sarcasm and context, and it treats news headlines as opinions. Combine it with filtering and engagement weighting, as shown above.

What does the VADER compound score mean?

It's a normalised score from -1 (most negative) to +1 (most positive). The usual thresholds are ≥ 0.05 for positive, ≤ -0.05 for negative and anything in between neutral.

Can I do sentiment analysis on Twitter in real time?

Close to it. Schedule the Tweet Scraper hourly with "only new items", score each batch, and alert on a drop in the weighted score. The search route runs a few hours behind live X, so for minute-by-minute monitoring of your own account, read its profile timeline instead.