Hot take: if I wanted to gather data via the internet, and I’m writing scripts/code to speed up the process, I have to follow some basic rules (ie look at the sitemap, find relevant robots.txt, follow that websites preference and rules). But it seems …