--- title: "How to Scrape Job Postings: Job Scraping Guide | Scrape.ly" description: "Scrape job postings from LinkedIn, Indeed and Google Jobs, merge them into one deduplicated feed with Python, schedule it daily, and see the real costs." url: https://scrape.ly/blog/job-scraping markdown: https://scrape.ly/blog/job-scraping.md --- [Use cases](https://scrape.ly/blog/category/use-cases.md) # Job Scraping: How to Scrape Job Postings From LinkedIn, Indeed and Google Jobs Job scraping collects public job ads from boards like LinkedIn, Indeed and Google Jobs into one table. Real runs from 22 and 23 October 2026, a Python script that merges and deduplicates the three sources, a daily no-code feed, and the cost per 1,000 jobs. [Openrole Actor publisher on the Scrape.ly Store](https://scrape.ly/blog/author/openrole.md) - Published Oct 23, 2026 - Reading time 9 min read ![Pixel-art robot with an orange >\_ on its terminal head takes a job card off a wide cork job board whose header shows the Google G, LinkedIn and Indeed logos, above six pinned job cards](https://scrape.ly/media/job-scraping.png?w=1080) To scrape job postings, you run a scraper per job board (LinkedIn, Indeed, Google Jobs) with the same search, map each board's fields to one schema, and remove the postings that appear on more than one board. Job scraping with ready-made Actors takes minutes: you pick the role and location, and each run returns structured rows with title, company, location, date, salary when published and the link to apply. This guide compares what each board gives you, using real runs from 22 and 23 October 2026, then builds one deduplicated feed in Python, schedules it without code, and shows what it costs per 1,000 jobs. ## What you can collect from each job board | Board | Actor | Key fields | Salary | Price per 1,000 jobs | | ----------- | ---------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | -------------------- | -------------------- | | LinkedIn | [LinkedIn Jobs Scraper](https://scrape.ly/store/openrole/linkedin-jobs-scraper.md) | title, company, location, posted date, seniority, employment type, applicants count, description, company website and size | when the ad shows it | $1.50 | | Indeed | [Indeed Scraper](https://scrape.ly/store/openrole/indeed-scraper.md) | title, company, location, salary text, job type, setting, experience level, company rating, description, external apply link | as printed on Indeed | $3.50 | | Google Jobs | [Google Jobs Scraper](https://scrape.ly/store/openrole/google-jobs-scraper.md) | title, company, location, `via` (the board Google credits), highlights such as qualifications and benefits, apply options | when Google shows it | $4.00 | All three read public pages without a login and run on Scrape.ly's proxies. The Store also has Actors for Glassdoor jobs and Upwork job posts; browse the [Store](https://scrape.ly/store.md) if you need a job scraper for another board. ## What job scraping is used for - **Labor market research.** Count openings by title, city and seniority, and watch the numbers move month to month. - **Competitor hiring watch.** A company that suddenly posts ten data engineering roles is building something. Track your rivals' openings on a schedule. - **Salary benchmarking.** Posted ranges, where boards show them, give you real numbers for offers and budgets. - **Job boards and aggregators.** Niche boards fill their listings from the big ones, with a link back to apply. - **Sales prospecting.** A company hiring for a role is often buying the tools that role uses. Hiring signals make strong lead lists. - **Recruiting.** Agencies see which employers hire for the profiles they place, and how fast ads are filled and reposted. ## Real runs: the same search on LinkedIn and Indeed On 22 October 2026 the LinkedIn Jobs Scraper's preset "Data engineer jobs in Berlin this week, 10 jobs" returned 10 jobs in 46 seconds for $0.0181. On 23 October we ran the Indeed Scraper with the same words, data engineer in Berlin, 10 jobs: 6 seconds, $0.0359. The first five rows of each: | LinkedIn: title | Company | Posted | Seniority | Applicants | | ------------------------------------------------------------- | ------------------------- | ---------- | ---------------- | ---------- | | Domain Data Engineer - Power Grid & System Operations (w/m/d) | 50Hertz Transmission GmbH | 2026-10-22 | Mid-Senior level | 56 | | Data Engineer - Digital Health Excellence Center (w/m/d) | EY | 2026-10-19 | Mid-Senior level | not shown | | Data Engineer (all genders) in Berlin - Jobbird.com | Jobster | 2026-10-22 | Not Applicable | not shown | | Data Engineer (f/m/x) | Netconomy | 2026-10-21 | Mid-Senior level | 50 | | Senior Data Engineer - Shop Analytics (all genders) | ABOUT YOU | 2026-10-16 | Mid-Senior level | 73 | | Indeed: title | Company | Posted | Salary | | -------------------------------------------------------------------------------------------- | ----------- | ---------- | --------- | | Data Engineer | MSA Safety | 2026-10-22 | not shown | | Senior Backend Engineer (f/m/x) | LiveEO GmbH | 2026-10-22 | not shown | | Analytics Engineer - All Genders | Doodle | 2026-10-22 | not shown | | Tech Lead - Craftnote | Aceve | 2026-10-22 | not shown | | Machine Learning Intern, Generative SI Applications (m/f/d) - Gigafactory Berlin-Brandenburg | Tesla | 2026-10-22 | not shown | The two boards answered the same question very differently, and that's the case for scraping more than one: - **Different employers.** The two lists had no company in common. Each board reaches employers the other misses, so a single-board feed undercounts the market. - **Different relevance.** All 10 LinkedIn titles contained "data". On Indeed, only 2 of 10 did (Data Engineer at MSA Safety, Analytics Engineer at Doodle); the rest were related engineering roles. Filter titles after you merge. - **Salary depends on the market.** One of the 10 Berlin Indeed rows had a salary (78.000 € – 88.000 € pro Jahr). In the Indeed run for our [Indeed scraping guide](https://scrape.ly/blog/scrape-indeed.md) (data analysts in New York, 23 October), all 10 rows had one. None of the 10 LinkedIn rows carried a salary. Google Jobs adds a third view, and the reason you must deduplicate. In the run for our [Google Jobs guide](https://scrape.ly/blog/scrape-google-jobs.md) (software engineers in San Francisco, 23 October, 10 jobs in 13 seconds, $0.0808), the `via` field showed where each posting came from: 3 via LinkedIn, 1 via Indeed, 3 via Uber's own careers site, and one each via ZipRecruiter, Callings.ai and a company careers page. A feed that scrapes LinkedIn, Indeed and Google Jobs for the same search sees some postings twice. ## How to build one deduplicated job feed with Python The approach: map every board to the same seven fields, then build a key from the normalized company, title and city. German ads add gender tags such as "(w/m/d)" or "(f/m/x)" that differ by board for the same job, so the normalizer strips them before comparing. Create an API token under **Settings → API & Integrations**, export it as `SCRAPELY_TOKEN` and `pip install scrapely-client-python`: ```python import csv import os import re import unicodedata from scrapely_client import ScrapelyClient client = ScrapelyClient(os.environ["SCRAPELY_TOKEN"], api_url="https://api.scrape.ly") GENDER_TAGS = re.compile(r"\((?:[mwfdxi]/?){2,4}\)|\((?:all genders?|all identities)\)|-\s*all genders", re.I) def run_items(actor, run_input): run = client.actor(actor).call(run_input=run_input) return list(client.dataset(run.default_dataset_id).iterate_items()) def norm(text): text = unicodedata.normalize("NFKD", text or "").encode("ascii", "ignore").decode().lower() return re.sub(r"[^a-z0-9]+", " ", GENDER_TAGS.sub("", text)).strip() rows = [] for j in run_items("openrole/linkedin-jobs-scraper", {"keywords": "data engineer", "location": "Berlin", "datePosted": "pastWeek", "limitPerSource": 10}): rows.append({"source": "LinkedIn", "title": j["title"], "company": j["companyName"], "location": j["location"], "posted": j.get("postedAt"), "salary": None, "url": j["link"]}) for j in run_items("openrole/indeed-scraper", {"position": "data engineer", "location": "Berlin", "country": "DE", "maxItemsPerSearch": 10}): rows.append({"source": "Indeed", "title": j["positionName"], "company": j["company"], "location": j["location"], "posted": (j.get("postingDateParsed") or "")[:10], "salary": j.get("salary"), "url": j["url"]}) for j in run_items("openrole/google-jobs-scraper", {"query": "data engineer", "location": "Berlin, Germany", "country": "de", "maxResults": 10}): ext = j.get("detected_extensions") or {} rows.append({"source": f"Google Jobs via {j.get('via')}", "title": j["title"], "company": j["company_name"], "location": j["location"], "posted": ext.get("posted_at"), "salary": ext.get("salary"), "url": j.get("share_link")}) feed = {} for row in rows: key = (norm(row["company"]), norm(row["title"]), norm(row["location"]).split(" ")[0]) if key in feed: feed[key]["also_on"] += f" {row['source']};" else: feed[key] = {**row, "also_on": ""} relevant = [r for r in feed.values() if re.search(r"\bdata\b|analytics", r["title"], re.I)] with open("jobs.csv", "w", newline="") as f: writer = csv.DictWriter(f, fieldnames=["source", "title", "company", "location", "posted", "salary", "url", "also_on"]) writer.writeheader() writer.writerows(relevant) print(f"{len(rows)} postings, {len(feed)} unique, {len(relevant)} relevant") ``` We tested the merge step on the saved LinkedIn and Indeed runs above: 20 postings, 20 unique (the boards shared no employer that day), and 12 with "data" or "analytics" in the title. When we fed it a LinkedIn "Data Engineer (f/m/x)" and an Indeed "Data Engineer (m/w/d)" at the same company in Berlin, it kept one row and recorded Indeed under `also_on`. Google Jobs reports the posting age as text ("2 days ago"), so store your own scrape date next to it. For a production feed, write the rows to a database keyed on the same tuple and update `also_on` instead of inserting. AI agents can do the same over MCP: add `https://api.scrape.ly/mcp` to Cursor or Claude Code and ask "Use scrapely to scrape data engineer jobs in Berlin from LinkedIn, Indeed and Google Jobs, 10 each, merge them and remove duplicates." ## Run it from the API without the client The LinkedIn run fetches each job's description and company page, which took 46 seconds for 10 jobs, so use the asynchronous endpoints: ```shell # 1. start the run; the response's data.id is the run ID curl -X POST "https://api.scrape.ly/v2/actors/openrole~linkedin-jobs-scraper/runs" -H "Authorization: Bearer $SCRAPELY_TOKEN" -H "Content-Type: application/json" -d '{"keywords": "data engineer", "location": "Berlin", "datePosted": "pastWeek", "limitPerSource": 10}' # 2. wait (up to 60 s per call; repeat until status is SUCCEEDED) curl "https://api.scrape.ly/v2/actor-runs/RUN_ID?waitForFinish=60" -H "Authorization: Bearer $SCRAPELY_TOKEN" # 3. read the items as CSV curl "https://api.scrape.ly/v2/actor-runs/RUN_ID/dataset/items?format=csv" -H "Authorization: Bearer $SCRAPELY_TOKEN" ``` The Indeed run took 6 seconds, short enough for the synchronous endpoint: ```shell curl -X POST "https://api.scrape.ly/v2/actors/openrole~indeed-scraper/run-sync-get-dataset-items" -H "Authorization: Bearer $SCRAPELY_TOKEN" -H "Content-Type: application/json" -d '{"position": "data engineer", "location": "Berlin", "country": "DE", "maxItemsPerSearch": 10}' ``` Scrape.ly Store LinkedIn Jobs Scraper Public LinkedIn job search by keyword and location: title, company, date, seniority, applicants and the full description, no login. $1.50 per 1,000 jobs. [Try it free](https://scrape.ly/store/openrole/linkedin-jobs-scraper.md) [Browse the Store](https://scrape.ly/store.md) ## Schedule a daily job feed without code 1. **Save one input per board.** On each Actor's page, set the search (keywords and location, a date filter such as **Date posted** past 24 hours on LinkedIn or **Posted within** 24h on Google Jobs) and use **Save as new…** in the **Input** menu. 2. **Create the automation.** Open **Automations**, click **New automation**, set **When** to **On a schedule** (daily at 9:00 is the default) and add the three saved inputs under **Run**. They start at the same time. 3. **Keep only new postings.** Under **Keep**, choose **Only new items since the last run** with the key fields `id, job_id`: LinkedIn and Indeed rows carry `id`, Google Jobs rows carry `job_id`. 4. **Summarize (optional).** Turn on the AI step in **Summarize** mode with an instruction like "List the new roles by company, with the salary when it's posted." It's billed per use, like the AI chat. 5. **Send it.** Post the summary to Slack and append the rows to Google Sheets to build a history. The [automations guide](https://scrape.ly/docs/automations.md) lists every option. Set **Max cost per run** on each run, and the run stops when it reaches it. ## What it costs Each Actor charges a fixed fee per job, plus small platform usage. Our runs: - LinkedIn Jobs: 10 jobs × $0.0015 = $0.015 in result fees; $0.0181 charged in total (46 seconds). - Indeed: 10 jobs × $0.0035 = $0.035; $0.0359 charged in total (6 seconds). - Google Jobs: 10 jobs × $0.004 = $0.04; $0.0808 charged in total (13 seconds; the rest is browser time and proxy). A worked example: 100 new jobs a day from each board is 100 × ($0.0015 + $0.0035 + $0.004) = $0.90 a day in result fees, $27 for 30 days, before platform usage. The Free plan's $5 of monthly credit covers a smaller pilot; the Starter plan at $19 a month includes $19 of credit. ## What to watch out for - **Duplicates and reposts.** Google Jobs aggregates other boards, and employers repost the same ad. Deduplicate on normalized company, title and city, and keep the first-seen date. - **Relevance drift.** Boards interpret keywords differently, as the Indeed run shows. Filter titles, or use the board's own filters in a search URL. - **Search caps.** Boards stop showing results deep into a search: about 1,000 jobs on LinkedIn and 600 to 750 on Indeed. Slice big searches by city or date; on LinkedIn, the Actor's **Split search by city locations** option does it for you, and on Google Jobs you run one query per city. - **Expired ads.** Indeed rows carry an `isExpired` flag; drop expired rows before you publish or count them. - **Personal data.** Job ads sometimes name a recruiter or include a contact email. Recruiters are private persons; keep the employer and drop the rest unless you need it. - **Blocking.** Job boards limit bursts. The Actors run on Scrape.ly's proxies, so you never manage IPs; daily runs are plenty for most feeds. ## Is it legal to scrape job postings? Job ads are published to be seen, and the Actors above read only public pages without logging in. In the United States, the Ninth Circuit held in [hiQ Labs v. LinkedIn](https://cdn.ca9.uscourts.gov/datastore/opinions/2022/04/18/17-16783.pdf) (April 2022) that the anti-hacking law's concept of "without authorization" "does not apply to public websites", while noting that other claims, such as breach of contract or copyright, "may also lie". The case itself later ended in a consent judgment against hiQ, so site terms still matter. Two practical rules follow. Job descriptions are written text, so republishing them in full on your own board needs more care than analyzing them; and recruiter names and emails are personal data under privacy laws like GDPR. This is not legal advice. Our guide on [whether web scraping is legal](https://scrape.ly/blog/is-web-scraping-legal.md) covers the cases in detail. ## Why Scrape.ly - **Three boards, one schema away.** LinkedIn, Indeed and Google Jobs Actors from one publisher, plus Glassdoor and Upwork, so adding a board is one more run step. - **Cents per hundred jobs.** $1.50 to $4.00 per 1,000 jobs, a maximum charge per run and a prepaid balance. - **Feeds, not one-off exports.** Schedules, "only new items", an AI summary step and deliveries to Slack, Google Sheets and webhooks come with the platform. - **Ready for agents.** The same Actors run from Claude, Cursor or ChatGPT over the hosted MCP server. ## FAQ ### What is job scraping? Job scraping is the automated collection of public job ads, such as title, company, location, date, salary and description, from job boards and career sites into a structured table. Teams use it for market research, hiring signals, salary data and job aggregators. ### Can I scrape LinkedIn jobs without logging in? Yes. LinkedIn's public job search works without an account, and the LinkedIn Jobs Scraper reads that public version. Our [LinkedIn jobs guide](https://scrape.ly/blog/scrape-linkedin-jobs.md) walks through the input, a real run and the output fields. ### How do I scrape Indeed? Enter a position, a location and the Indeed country site in the Indeed Scraper, or paste Indeed search and company URLs, and set the maximum listings per search. Our [Indeed guide](https://scrape.ly/blog/scrape-indeed.md) covers it step by step with salary data. ### Can I scrape Google Jobs? Yes. The Google Jobs Scraper takes a role, a location, a country and an optional "posted within" filter, and returns the jobs with the board Google credits for each one. See [how to scrape Google Jobs](https://scrape.ly/blog/scrape-google-jobs.md) for a full run. ### Is there a free job scraper? The Free plan includes $5 of usage credit every month, no credit card needed. At $1.50 per 1,000 LinkedIn jobs, that's enough for a few thousand postings before platform usage. ### How often should I scrape job postings? Daily suits most feeds, with a "posted in the last 24 hours" filter so each run only fetches new ads. Competitor hiring watches can run weekly. Scrape.ly Store Indeed Scraper Indeed jobs by title and location in 60+ countries: salary text, job type, company rating, description and apply link. $3.50 per 1,000 listings. [Try it free](https://scrape.ly/store/openrole/indeed-scraper.md) [Browse the Store](https://scrape.ly/store.md) About the author [Openrole](https://scrape.ly/blog/author/openrole.md) Actor publisher on the Scrape.ly Store Every open role, wherever it is posted: job boards, company ATS pages and freelance markets in dozens of countries. For recruiters, labour-market analysts and job aggregators. A Scrapely studio. - [Website](https://scrape.ly/users/openrole.md) Keep reading ## More in Use cases - [![Pixel-art robot with an orange >\_ on its terminal head takes a yellow price tag off a store shelf whose header shows the Amazon and Walmart logos and drops it into a crate of price tags](https://scrape.ly/media/price-scraping.png?w=750)](https://scrape.ly/blog/price-scraping.md) ### [Price Scraping: How to Monitor Competitor Prices Automatically](https://scrape.ly/blog/price-scraping.md) [Price scraping collects product prices from retailer pages so you can compare them and react. Here is a real cross-retailer check of one keyboard on Amazon, Walmart, Best Buy and Target, a daily workflow with schedules and Google Sheets, Python code, and the cost per check.](https://scrape.ly/blog/price-scraping.md) - [![Pixel-art robot with an orange >\_ on its terminal head carries a lead card with an avatar and contact lines to a crate of leads, with cables running to it from the Google Maps pin and from a sign bearing the blue LinkedIn logo](https://scrape.ly/media/lead-generation-scraping.png?w=750)](https://scrape.ly/blog/lead-generation-scraping.md) ### [Lead Scraping: Build B2B Lead Lists From Public Business Data](https://scrape.ly/blog/lead-generation-scraping.md) [Lead scraping turns public business listings and company websites into a lead list. Here is a real run (5 Austin web design agencies, $0.025), a no-code workflow, Python that merges Google Maps, websites and LinkedIn company data, and the GDPR and CAN-SPAM rules for outreach.](https://scrape.ly/blog/lead-generation-scraping.md) - [![Pixel-art robot with an orange >\_ on its terminal head pulls a social post card with a heart out of a wide monitor showing the Instagram, TikTok, X, YouTube and Reddit logos](https://scrape.ly/media/social-media-scraping.png?w=750)](https://scrape.ly/blog/social-media-scraping.md) ### [Social Media Scraping: Platforms, Data and the Rules](https://scrape.ly/blog/social-media-scraping.md) [Social media scraping collects public posts, videos and engagement counts from Instagram, TikTok, X, YouTube, Reddit, LinkedIn and Facebook. What each platform gives you, prices per 1,000 results, real runs from five platforms, one Python script for all of them, and the privacy rules.](https://scrape.ly/blog/social-media-scraping.md) ## The internet is full of data. Start owning yours. Scrape.ly gives you the infrastructure, the tooling, and the team to extract, process, and operationalize web data at any scale. Setup takes minutes. The advantage lasts forever. [Get Started Free](https://my.scrape.ly/register) [Chat with Support](mailto:support@scrape.ly?subject=Question%20about%20Scrape.ly\&body=Hi%2C%20I%20have%20a%20question%20about%20Scrape.ly.) No credit card. No setup calls.