Is web scraping legal? What matters, in plain words

Reading public pages is often allowed, but not always, and the details depend on where you are and what you collect. These are the points that come up again and again. For your own case, ask a lawyer.

Facts, not legal advice

Usually fine
public pages, facts
slow, polite, nothing personal
Ask first
logins, personal data
whole databases, terms that forbid it

Nederlands

What usually decides it

  1. Public or behind a login. Pages anyone can open are a different case from pages you need an account for. Never scrape behind a login you were not given for that.

  2. Personal data. Names, emails and profiles fall under the GDPR in Europe, also when they are public. Collect only what you need, or nothing personal at all.

  3. The site's terms. Terms of use can forbid scraping. In the EU, the Ryanair v PR Aviation ruling (2015) said such terms can bind you when a database has no other protection.

  4. Database rights. In the EU, copying a substantial part of a database someone invested in can break the database right, even when each fact is free.

  5. Load on the site. Hammering a server can cause damage, and claims. Go slow, cache, and fetch only what changed.

And in the US?

In hiQ Labs v. LinkedIn, a US appeals court found that scraping public pages is unlikely to be "access without authorization" under the CFAA (2022). The case still ended with hiQ losing on LinkedIn's terms of use. Public is not the same as free to take.

Websites that allow web scraping

  1. robots.txt. Says which paths a site asks bots to skip. Not a law, but a clear signal: respect it.

  2. An official API or feed. If the site offers one, use it. It is the clearest permission there is.

  3. Open data. Government and research sites often publish their data under an open licence.

More on this

Try it on your own pages

Paste a few links and a couple of values. The look is free, and you hear the price before anything is due.

Try it on your pages