Yeah this is one of those things that hurts some real users and AI scrapers have figured out how to bypass. Not worth it IMO.
Yeah this is one of those things that hurts some real users and AI scrapers have figured out how to bypass. Not worth it IMO.
How do you figure it hurts real users? The amount of compute/energy used on the proof of work is pretty minimal. You're using more when you watch a YouTube video or browse a JS-heavy web app.
Of course a sophisticated scraper can "figure out" how to bypass. It isn't trying to be foolproof, it's adding an extra cost to deter massive amounts of bot traffic.
I put it in front of my hobby project because I can't afford to serve hundreds of thousands of bot requests from residential proxies all across the world, and I didn't want to route all of my traffic through a third party company like Cloudflare.
I've been happy with Anubis.
1. It makes the web more annoying and time wasting.
2. I'm a real user and I have to enable JavaScript and run arbitrary programs from untrusted websites to access them. One browser that I use doesn't even support JavaScript (w3m). Fortunately it is unaffected because it doesn't have the string "Mozilla" in its UA and Anubis lets every user without that pass to not hurt legitimate bots like search engine crawlers. As said Anubis is trivially bypassed. The only reason it isn't bypassed is because it's simply too irrelevant to the AI companies.
I empathize, I hate the bloated JS-heavy web and avoid it when I can. I just don't see a better solution. The options for me are: sacrifice some legit users by using Anubis, or sacrifice all users by shutting down my service because I can't afford to serve the bot traffic.
There are false positives. Every time I've used a tool like this, I get reports from my users about it blocking their access. It's especially bad for users who use privacy oriented browsers like Brave or use VPNs.
For a personal site where you don't care about 0.1% of your users getting blocked, it's probably fine. For a business, blocking some of your paying customers isn't acceptable.
> For a business, blocking some of your paying customers isn't acceptable.
That depends on your business and the costs you're incurring from the bots. It might be justifiable to lose 0.1% of your customers if it means saving server costs from serving useless bot traffic.
I suppose it would depend, yeah. I find having paying customers being blocked to be completely unacceptable. Not only is a bad customer experience, but they tend to be very vocal about it. Posting online, leaving bad reviews, etc.
Having to serve 1000x more traffic from bots is completely unacceptable for most site owners as well.
The fact that this trade-off isn't obvious indicates you haven't tried to run a site at scale and make a living off of it.
You are quite incorrect. I make a living off a website and 99.9% of my traffic is from bots. Literally hundreds of millions of requests from bots. The cost of bots is worth the cost of not harming legit users.