Web Scraping and GDPR: What You Can Collect and How

GDPR web scraping questions come down to one thing: whether the data you collect is personal data. Prices, product specs, job ads without names, company facts and search rankings usually aren't, and the GDPR has little to say about them. Names, usernames, profile photos, reviews tied to a person, contact details and anything else that relates to an identifiable person are, and from the moment you collect them the GDPR applies in full, even if the data was public.

This guide explains what counts as personal data, why "it was public" isn't a legal basis, what the Dutch and French data protection authorities have said about scraping, the safeguards regulators expect, an enforcement example, and a checklist you can apply before your next run. It quotes the regulation and the regulators directly, read on 27 October 2026.

What counts as personal data

The GDPR's definition in Article 4(1) is broad: "'personal data' means any information relating to an identified or identifiable natural person ('data subject'); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier".

For scrapers, that means:

• Obviously personal: names, email addresses, phone numbers, profile pictures, usernames and handles, home addresses.
• Personal in context: a review with the reviewer's display name, a comment thread, a job title together with a company, a listing agent on a property page, a seller name on a marketplace.
• Usually not personal: product prices and stock, hotel ratings, company revenue, search results, aggregated counts, text from corporate pages.

Two tests help. Could you, or anyone, link this row to a person? And do you actually need that link for your purpose? If the answer to the second is no, don't collect the field.

Public data still needs a legal basis

The most common misconception is that public data is free to use. Under the GDPR, every processing of personal data needs one of the legal bases in Article 6, and collecting is processing. For companies scraping the web, the only realistic basis is usually Article 6(1)(f): "processing is necessary for the purposes of the legitimate interests pursued by the controller or by a third party, except where such interests are overridden by the interests or fundamental rights and freedoms of the data subject which require protection of personal data, in particular where the data subject is a child."

The Dutch Data Protection Authority (Autoriteit Persoonsgegevens) says the same in its scraping guidance for private organisations (April 2025, in Dutch). In translation: if you want to use scraping or scraped data as a private organisation, you can probably rely only on legitimate interest, and "you also need this basis if you only collect information that is already on the internet and publicly accessible to everyone". The original reads: "Deze grondslag heeft u óók nodig als u alleen informatie verzamelt die al op internet staat en die voor iedereen openbaar toegankelijk is."

The same authority had read "legitimate interest" narrowly, and changed that after a ruling of the EU Court of Justice. Its changes page says a purely commercial interest "can qualify as a legitimate interest and the AP's interpretation was too strict", while "The Court does emphasise that there are two other requirements for this legal basis." Those two requirements are the ones that decide most scraping projects.

The legitimate interest test, applied to scraping

Legitimate interest has three parts. Write down your answers before you scrape; regulators call this a legitimate interest assessment.

1. Purpose. Is your interest legitimate and specific? "Monitor what customers say about our product in public reviews" is. "Collect everything in case it's useful" isn't.
2. Necessity. Do you need the personal data for that purpose, and only that much? Review text and star rating may be necessary; the reviewer's name and photo usually aren't.
3. Balancing. Would the people concerned reasonably expect this use, and could it harm them? Scraping professional profiles to email people at scale fails far more often than counting complaints in product reviews.

The Article 5(1)(c) principle of data minimisation runs through all three: personal data must be "adequate, relevant and limited to what is necessary".

What regulators expect from scrapers

France's CNIL published a focus sheet on web scraping in June 2025, written for building AI training data but useful for any scrape. Its mandatory measures, under the minimisation principle, include defining precise collection criteria in advance, excluding categories of data you don't need (through filters, or by leaving out sites that structurally contain such data), deleting irrelevant data "immediately after collection or as soon as it is identified as such", and excluding "websites that clearly oppose the scraping of their content for the purpose of creating training databases by using robots.txt exclusion protocols or implementing CAPTCHA". Our robots.txt and web scraping guide covers how to read those signals.

The Dutch guidance adds a warning about sensitive data. Scraping often sweeps up special categories of data, such as health, religion or political views (Article 9), even when you didn't intend to, and a private organisation will usually not be allowed to process them. If they're in your dataset, you generally can't keep using it until they're removed.

Telling people

Article 14 requires you to inform people whose data you collect from somewhere other than them. There's an exception where "the provision of such information proves impossible or would involve a disproportionate effort", which scrapers often rely on. It isn't automatic: you still need to publish the information (for example in your privacy notice) and to show why individual notice is disproportionate.

An enforcement example

On 3 September 2024 the Dutch authority fined Clearview AI €30.5 million. The company had built a database of more than 30 billion photos of faces, which it "scrapes ... automatically from the Internet", and turned them into biometric codes. The decision found violations of the lawfulness principle and Article 6 (no legal basis), Article 9 (biometric data), Articles 12 and 14 (people weren't informed) and Article 15 (access requests went unanswered). It's an extreme case, but every one of those findings applies to ordinary scraping too: basis, sensitive data, transparency and rights.

A GDPR checklist for web scraping

1. Decide whether you need personal data at all. Most market, price, product and SEO work doesn't. If you don't need it, configure the scraper not to collect it.
2. Write the purpose down, with your legitimate interest assessment, before the first run.
3. Collect the minimum. Pick fields, filter at the source, and drop names and contact details you won't use.
4. Pseudonymise early. Replace names with stable IDs if you need to count per person, and show [reviewer 1] instead of a name in anything you share. Our Google Play reviews tutorial shows the pattern.
5. Keep it short. Set a retention period and delete raw runs when the analysis is done.
6. Watch for sensitive data. Health forums, religious or political content and anything about children need a hard look, or exclusion.
7. Respect objections. Honour robots.txt opt-outs, and have a way to handle access and deletion requests.
8. Mind the other rules. The GDPR sits next to site terms, copyright and database rights; see is web scraping legal? for those.

How Scrape.ly helps you collect less

• Choose fields at download time. The dataset API takes fields= to download only the columns you name, and omit= to drop the ones you don't want, so names never reach your systems. For a review run, https://api.scrape.ly/v2/actor-runs/RUN_ID/dataset/items?format=csv&fields=rating,body,date,appVersion returns stars, text, date and version, and nothing about the reviewer.
• Short storage by default. Run storage expires on its own: after 2 days on the Free plan and 31 days on Starter, per the public plan data. Export what you need and let the rest lapse.
• Data you can describe. Every Store Actor documents its output fields, so you can tell in advance whether a run will contain personal data, and the security docs cover how the platform handles access.
• Most jobs don't need people at all. Price monitoring, SEO rank tracking and real estate data are about products, pages and properties; for reviews, review monitoring works on text and stars without names.

This guide is general information, not legal advice. Data protection depends on your purpose, your jurisdiction and the data you collect; for a commercial project, ask a lawyer or your data protection officer about your case.

FAQ

Is web scraping legal under the GDPR?

Scraping data that isn't personal is outside the GDPR. Scraping personal data can be lawful, usually on the basis of legitimate interest, if you have a specific purpose, collect only what's necessary, respect people's expectations and rights, and keep it secure. Regulators such as the Dutch authority consider broad scraping of personal data hard to justify.

Is publicly available data exempt from the GDPR?

No. The GDPR applies to personal data whether or not it's public. The Dutch authority states that you need a legal basis "also if you only collect information that is already on the internet and publicly accessible to everyone".

Do I need consent to scrape personal data?

Usually consent isn't practical, so scrapers rely on legitimate interest instead, which requires a documented balancing of your interest against the people's rights. Special categories of data, such as health or political views, have stricter rules.

Do I have to tell people I scraped their data?

Article 14 requires it, with an exception where informing each person "proves impossible or would involve a disproportionate effort". Even then you should publish the information, for example in your privacy notice.

Does the GDPR apply if my company is outside the EU?

It can. The GDPR applies to companies outside the EU that offer goods or services to people in the EU or monitor their behaviour there. Clearview AI, a US company with no EU office, was fined by the Dutch authority.

Is scraping company data a GDPR issue?

Company facts like revenue, address of the business or product lists aren't personal data. Data about the people at a company, such as names, roles and direct contact details, is.