Job Scraping: How to Scrape Job Postings From LinkedIn, Indeed and Google Jobs

To scrape job postings, you run a scraper per job board (LinkedIn, Indeed, Google Jobs) with the same search, map each board's fields to one schema, and remove the postings that appear on more than one board. Job scraping with ready-made Actors takes minutes: you pick the role and location, and each run returns structured rows with title, company, location, date, salary when published and the link to apply.

This guide compares what each board gives you, using real runs from 22 and 23 October 2026, then builds one deduplicated feed in Python, schedules it without code, and shows what it costs per 1,000 jobs.

What you can collect from each job board

All three read public pages without a login and run on Scrape.ly's proxies. The Store also has Actors for Glassdoor jobs and Upwork job posts; browse the Store if you need a job scraper for another board.

What job scraping is used for

• Labor market research. Count openings by title, city and seniority, and watch the numbers move month to month.
• Competitor hiring watch. A company that suddenly posts ten data engineering roles is building something. Track your rivals' openings on a schedule.
• Salary benchmarking. Posted ranges, where boards show them, give you real numbers for offers and budgets.
• Job boards and aggregators. Niche boards fill their listings from the big ones, with a link back to apply.
• Sales prospecting. A company hiring for a role is often buying the tools that role uses. Hiring signals make strong lead lists.
• Recruiting. Agencies see which employers hire for the profiles they place, and how fast ads are filled and reposted.

Real runs: the same search on LinkedIn and Indeed

On 22 October 2026 the LinkedIn Jobs Scraper's preset "Data engineer jobs in Berlin this week, 10 jobs" returned 10 jobs in 46 seconds for $0.0181. On 23 October we ran the Indeed Scraper with the same words, data engineer in Berlin, 10 jobs: 6 seconds, $0.0359. The first five rows of each:

The two boards answered the same question very differently, and that's the case for scraping more than one:

• Different employers. The two lists had no company in common. Each board reaches employers the other misses, so a single-board feed undercounts the market.
• Different relevance. All 10 LinkedIn titles contained "data". On Indeed, only 2 of 10 did (Data Engineer at MSA Safety, Analytics Engineer at Doodle); the rest were related engineering roles. Filter titles after you merge.
• Salary depends on the market. One of the 10 Berlin Indeed rows had a salary (78.000 € – 88.000 € pro Jahr). In the Indeed run for our Indeed scraping guide (data analysts in New York, 23 October), all 10 rows had one. None of the 10 LinkedIn rows carried a salary.

Google Jobs adds a third view, and the reason you must deduplicate. In the run for our Google Jobs guide (software engineers in San Francisco, 23 October, 10 jobs in 13 seconds, $0.0808), the via field showed where each posting came from: 3 via LinkedIn, 1 via Indeed, 3 via Uber's own careers site, and one each via ZipRecruiter, Callings.ai and a company careers page. A feed that scrapes LinkedIn, Indeed and Google Jobs for the same search sees some postings twice.

How to build one deduplicated job feed with Python

The approach: map every board to the same seven fields, then build a key from the normalized company, title and city. German ads add gender tags such as "(w/m/d)" or "(f/m/x)" that differ by board for the same job, so the normalizer strips them before comparing.

Create an API token under Settings → API & Integrations, export it as SCRAPELY_TOKEN and pip install scrapely-client-python:

import csv
import os
import re
import unicodedata
from scrapely_client import ScrapelyClient

client = ScrapelyClient(os.environ["SCRAPELY_TOKEN"], api_url="https://api.scrape.ly")
GENDER_TAGS = re.compile(r"\((?:[mwfdxi]/?){2,4}\)|\((?:all genders?|all identities)\)|-\s*all genders", re.I)

def run_items(actor, run_input):
    run = client.actor(actor).call(run_input=run_input)
    return list(client.dataset(run.default_dataset_id).iterate_items())

def norm(text):
    text = unicodedata.normalize("NFKD", text or "").encode("ascii", "ignore").decode().lower()
    return re.sub(r"[^a-z0-9]+", " ", GENDER_TAGS.sub("", text)).strip()

rows = []
for j in run_items("openrole/linkedin-jobs-scraper",
                   {"keywords": "data engineer", "location": "Berlin", "datePosted": "pastWeek", "limitPerSource": 10}):
    rows.append({"source": "LinkedIn", "title": j["title"], "company": j["companyName"], "location": j["location"],
                 "posted": j.get("postedAt"), "salary": None, "url": j["link"]})
for j in run_items("openrole/indeed-scraper",
                   {"position": "data engineer", "location": "Berlin", "country": "DE", "maxItemsPerSearch": 10}):
    rows.append({"source": "Indeed", "title": j["positionName"], "company": j["company"], "location": j["location"],
                 "posted": (j.get("postingDateParsed") or "")[:10], "salary": j.get("salary"), "url": j["url"]})
for j in run_items("openrole/google-jobs-scraper",
                   {"query": "data engineer", "location": "Berlin, Germany", "country": "de", "maxResults": 10}):
    ext = j.get("detected_extensions") or {}
    rows.append({"source": f"Google Jobs via {j.get('via')}", "title": j["title"], "company": j["company_name"],
                 "location": j["location"], "posted": ext.get("posted_at"), "salary": ext.get("salary"),
                 "url": j.get("share_link")})

feed = {}
for row in rows:
    key = (norm(row["company"]), norm(row["title"]), norm(row["location"]).split(" ")[0])
    if key in feed:
        feed[key]["also_on"] += f" {row['source']};"
    else:
        feed[key] = {**row, "also_on": ""}

relevant = [r for r in feed.values() if re.search(r"\bdata\b|analytics", r["title"], re.I)]
with open("jobs.csv", "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=["source", "title", "company", "location", "posted", "salary", "url", "also_on"])
    writer.writeheader()
    writer.writerows(relevant)
print(f"{len(rows)} postings, {len(feed)} unique, {len(relevant)} relevant")

We tested the merge step on the saved LinkedIn and Indeed runs above: 20 postings, 20 unique (the boards shared no employer that day), and 12 with "data" or "analytics" in the title. When we fed it a LinkedIn "Data Engineer (f/m/x)" and an Indeed "Data Engineer (m/w/d)" at the same company in Berlin, it kept one row and recorded Indeed under also_on.

Google Jobs reports the posting age as text ("2 days ago"), so store your own scrape date next to it. For a production feed, write the rows to a database keyed on the same tuple and update also_on instead of inserting.

AI agents can do the same over MCP: add https://api.scrape.ly/mcp to Cursor or Claude Code and ask "Use scrapely to scrape data engineer jobs in Berlin from LinkedIn, Indeed and Google Jobs, 10 each, merge them and remove duplicates."

Run it from the API without the client

The LinkedIn run fetches each job's description and company page, which took 46 seconds for 10 jobs, so use the asynchronous endpoints:

# 1. start the run; the response's data.id is the run ID
curl -X POST "https://api.scrape.ly/v2/actors/openrole~linkedin-jobs-scraper/runs" \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"keywords": "data engineer", "location": "Berlin", "datePosted": "pastWeek", "limitPerSource": 10}'
# 2. wait (up to 60 s per call; repeat until status is SUCCEEDED)
curl "https://api.scrape.ly/v2/actor-runs/RUN_ID?waitForFinish=60" \
  -H "Authorization: Bearer $SCRAPELY_TOKEN"
# 3. read the items as CSV
curl "https://api.scrape.ly/v2/actor-runs/RUN_ID/dataset/items?format=csv" \
  -H "Authorization: Bearer $SCRAPELY_TOKEN"

The Indeed run took 6 seconds, short enough for the synchronous endpoint:

curl -X POST "https://api.scrape.ly/v2/actors/openrole~indeed-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $SCRAPELY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"position": "data engineer", "location": "Berlin", "country": "DE", "maxItemsPerSearch": 10}'

Schedule a daily job feed without code

1. Save one input per board. On each Actor's page, set the search (keywords and location, a date filter such as Date posted past 24 hours on LinkedIn or Posted within 24h on Google Jobs) and use Save as new… in the Input menu.
2. Create the automation. Open Automations, click New automation, set When to On a schedule (daily at 9:00 is the default) and add the three saved inputs under Run. They start at the same time.
3. Keep only new postings. Under Keep, choose Only new items since the last run with the key fields id, job_id: LinkedIn and Indeed rows carry id, Google Jobs rows carry job_id.
4. Summarize (optional). Turn on the AI step in Summarize mode with an instruction like "List the new roles by company, with the salary when it's posted." It's billed per use, like the AI chat.
5. Send it. Post the summary to Slack and append the rows to Google Sheets to build a history. The automations guide lists every option.

Set Max cost per run on each run, and the run stops when it reaches it.

What it costs

Each Actor charges a fixed fee per job, plus small platform usage. Our runs:

• LinkedIn Jobs: 10 jobs × $0.0015 = $0.015 in result fees; $0.0181 charged in total (46 seconds).
• Indeed: 10 jobs × $0.0035 = $0.035; $0.0359 charged in total (6 seconds).
• Google Jobs: 10 jobs × $0.004 = $0.04; $0.0808 charged in total (13 seconds; the rest is browser time and proxy).

A worked example: 100 new jobs a day from each board is 100 × ($0.0015 + $0.0035 + $0.004) = $0.90 a day in result fees, $27 for 30 days, before platform usage. The Free plan's $5 of monthly credit covers a smaller pilot; the Starter plan at $19 a month includes $19 of credit.

What to watch out for

• Duplicates and reposts. Google Jobs aggregates other boards, and employers repost the same ad. Deduplicate on normalized company, title and city, and keep the first-seen date.
• Relevance drift. Boards interpret keywords differently, as the Indeed run shows. Filter titles, or use the board's own filters in a search URL.
• Search caps. Boards stop showing results deep into a search: about 1,000 jobs on LinkedIn and 600 to 750 on Indeed. Slice big searches by city or date; on LinkedIn, the Actor's Split search by city locations option does it for you, and on Google Jobs you run one query per city.
• Expired ads. Indeed rows carry an isExpired flag; drop expired rows before you publish or count them.
• Personal data. Job ads sometimes name a recruiter or include a contact email. Recruiters are private persons; keep the employer and drop the rest unless you need it.
• Blocking. Job boards limit bursts. The Actors run on Scrape.ly's proxies, so you never manage IPs; daily runs are plenty for most feeds.

Is it legal to scrape job postings?

Job ads are published to be seen, and the Actors above read only public pages without logging in. In the United States, the Ninth Circuit held in hiQ Labs v. LinkedIn (April 2022) that the anti-hacking law's concept of "without authorization" "does not apply to public websites", while noting that other claims, such as breach of contract or copyright, "may also lie". The case itself later ended in a consent judgment against hiQ, so site terms still matter.

Two practical rules follow. Job descriptions are written text, so republishing them in full on your own board needs more care than analyzing them; and recruiter names and emails are personal data under privacy laws like GDPR. This is not legal advice. Our guide on whether web scraping is legal covers the cases in detail.

Why Scrape.ly

• Three boards, one schema away. LinkedIn, Indeed and Google Jobs Actors from one publisher, plus Glassdoor and Upwork, so adding a board is one more run step.
• Cents per hundred jobs. $1.50 to $4.00 per 1,000 jobs, a maximum charge per run and a prepaid balance.
• Feeds, not one-off exports. Schedules, "only new items", an AI summary step and deliveries to Slack, Google Sheets and webhooks come with the platform.
• Ready for agents. The same Actors run from Claude, Cursor or ChatGPT over the hosted MCP server.

FAQ

What is job scraping?

Job scraping is the automated collection of public job ads, such as title, company, location, date, salary and description, from job boards and career sites into a structured table. Teams use it for market research, hiring signals, salary data and job aggregators.

Can I scrape LinkedIn jobs without logging in?

Yes. LinkedIn's public job search works without an account, and the LinkedIn Jobs Scraper reads that public version. Our LinkedIn jobs guide walks through the input, a real run and the output fields.

How do I scrape Indeed?

Enter a position, a location and the Indeed country site in the Indeed Scraper, or paste Indeed search and company URLs, and set the maximum listings per search. Our Indeed guide covers it step by step with salary data.

Can I scrape Google Jobs?

Yes. The Google Jobs Scraper takes a role, a location, a country and an optional "posted within" filter, and returns the jobs with the board Google credits for each one. See how to scrape Google Jobs for a full run.

Is there a free job scraper?

The Free plan includes $5 of usage credit every month, no credit card needed. At $1.50 per 1,000 LinkedIn jobs, that's enough for a few thousand postings before platform usage.

How often should I scrape job postings?

Daily suits most feeds, with a "posted in the last 24 hours" filter so each run only fetches new ads. Competitor hiring watches can run weekly.