The story has not changed much. Every so often I will remove most of the blocks put in place, and within a few hours I'm back to having to block them. Many of the cheaper VPNs are also hosted on AWS / Google Cloud / Azure (or other cloud providers), which are also unilaterally blocked.
I would much prefer we did not have to do this, but it is what it is.
VPNs even from big public providers have not been a reliable way to protect anonymity for a while now. Use VPNs for cryptographic security and circumventing region control.
The problem is the whack-a-mole game with hackers and script kiddies. It used to be the case that banning known colo ASNs was enough to get rid of nuisance by STROs, then there was a flood of hacked routers being used for DDoS that was really annoying to get rid of, and then came "residential IP" VPNs and commercial VPNs, both of which get routinely abused by AI scrapers and frankly, the AI scrapers are a worse enemy than the skiddies of 10 years ago. They ruin everything.
And you as a site operator can't really tell apart skiddies, griefers, AI scrapers and legitimate users apart any more.
In what I have seen personally, creating absurdly more load by hitting "expensive" pages that no normal user would ever click in that frequency. The AI scraper bots are really, really dumb - they just follow everything that looks like a link.
Another particularly annoying thing was when spam bots got brainy enough (if I were to guess with AI?) that managed to bypass our maths captcha. That one really still pisses me off because I don't like to torture users or having to use GDPR-violating services.
If all the traffic you see from a particular netblock is people posting hate speech, you're probably not losing much by dropping everything from that whole range.
> Almost as if you shouldn't be banning users because of their IP unless that IP specifically has openly attacked you.
There is no net benefit to allowing non-residential IP addresses by default, maybe add the Google search indexer to the exception list. And with residential IP addresses, unless you're international, it doesn't make sense to allow regions other than your target markets.
The only way to deal with the bot traffic plagueing the modern internet is to cut off as much traffic as you reasonably can.
I've never seen anyone using a VPN for anything other than disruptive behaviour. I had to block vast swathes of mobile broadband providers in a certain warlike Middle Eastern country because if I didn't I'd have anywhere from 100 to 1000 new users every single morning who'd all posted hate speech that won't post here for fear of triggering the right-wing apologists.
Now they just do that over VPNs, which makes keeping them out all the more difficult.
Unfortunately true. I wrote about it early last year here: https://blog.xkeeper.net/uncategorized/tcrf-has-been-getting...
The story has not changed much. Every so often I will remove most of the blocks put in place, and within a few hours I'm back to having to block them. Many of the cheaper VPNs are also hosted on AWS / Google Cloud / Azure (or other cloud providers), which are also unilaterally blocked.
I would much prefer we did not have to do this, but it is what it is.
This is why you should self host your VPN.
But where do you self-host it? Most sites that block VPNs also block VPSes
Does that not defeat the anonymity aspect?
VPNs even from big public providers have not been a reliable way to protect anonymity for a while now. Use VPNs for cryptographic security and circumventing region control.
You mean pseudo anonymity, from advertisers mostly?
Many of us only like legitimate users, and therefore block VPNs.
What makes a VPN user inherently “illegitimate” in your view?
The problem is the whack-a-mole game with hackers and script kiddies. It used to be the case that banning known colo ASNs was enough to get rid of nuisance by STROs, then there was a flood of hacked routers being used for DDoS that was really annoying to get rid of, and then came "residential IP" VPNs and commercial VPNs, both of which get routinely abused by AI scrapers and frankly, the AI scrapers are a worse enemy than the skiddies of 10 years ago. They ruin everything.
And you as a site operator can't really tell apart skiddies, griefers, AI scrapers and legitimate users apart any more.
What are they doing exactly?
In what I have seen personally, creating absurdly more load by hitting "expensive" pages that no normal user would ever click in that frequency. The AI scraper bots are really, really dumb - they just follow everything that looks like a link.
Another particularly annoying thing was when spam bots got brainy enough (if I were to guess with AI?) that managed to bypass our maths captcha. That one really still pisses me off because I don't like to torture users or having to use GDPR-violating services.
Almost as if you shouldn't be banning users because of their IP unless that IP specifically has openly attacked you.
Or I guess you can just DENY ALL.
If all the traffic you see from a particular netblock is people posting hate speech, you're probably not losing much by dropping everything from that whole range.
> Almost as if you shouldn't be banning users because of their IP unless that IP specifically has openly attacked you.
There is no net benefit to allowing non-residential IP addresses by default, maybe add the Google search indexer to the exception list. And with residential IP addresses, unless you're international, it doesn't make sense to allow regions other than your target markets.
The only way to deal with the bot traffic plagueing the modern internet is to cut off as much traffic as you reasonably can.
They're using a VPN.
I've never seen anyone using a VPN for anything other than disruptive behaviour. I had to block vast swathes of mobile broadband providers in a certain warlike Middle Eastern country because if I didn't I'd have anywhere from 100 to 1000 new users every single morning who'd all posted hate speech that won't post here for fear of triggering the right-wing apologists.
Now they just do that over VPNs, which makes keeping them out all the more difficult.
Well, you can just give me a list of the domains you operate, and I can put them in the network blacklist.