• LostXOR
    link
    fedilink
    02 days ago

    Even so I think it would be totally reasonable for them to block web scrapers, as they provide better ways to download all their data.

    • FaceDeer
      link
      fedilink
      02 days ago

      At the root of this comment chain is a proposal to have laws passed about this.

      People can set up their web servers however they like. It’s on them to do that, it’s their web servers. I don’t think there should be legislation about whether you’re allowed to issue perfectly ordinary HTTP requests to a public server, let the server decide how to respond to them.