What usually decides it
Public or behind a login. Pages anyone can open are a different case from pages you need an account for. Never scrape behind a login you were not given for that.
Personal data. Names, emails and profiles fall under the GDPR in Europe, also when they are public. Collect only what you need, or nothing personal at all.
The site's terms. Terms of use can forbid scraping. In the EU, the Ryanair v PR Aviation ruling (2015) said such terms can bind you when a database has no other protection.
Database rights. In the EU, copying a substantial part of a database someone invested in can break the database right, even when each fact is free.
Load on the site. Hammering a server can cause damage, and claims. Go slow, cache, and fetch only what changed.
And in the US?
In hiQ Labs v. LinkedIn, a US appeals court found that scraping public pages is unlikely to be "access without authorization" under the CFAA (2022). The case still ended with hiQ losing on LinkedIn's terms of use. Public is not the same as free to take.
Websites that allow web scraping
robots.txt. Says which paths a site asks bots to skip. Not a law, but a clear signal: respect it.
An official API or feed. If the site offers one, use it. It is the clearest permission there is.
Open data. Government and research sites often publish their data under an open licence.