Cloudflare is implementing the (still-emerging) Web Bot Auth standard. We're working on the same at Stytch for https://IsAgent.dev .
The discourse around this is a little wild and I'm glad you said this. The allowlist is a Cloudflare feature and their customers are free to use it. The core functionality involving HTTP Message Signatures is decentralized and open, so anyone can adopt it and benefit.
THANK YOU. The discourse on this is wild, people seem to be ranting agianst the Web Bot Auth standard without understanding what it is because of their (honestly quite legitimate) fears about Cloudflare's gatekeeping near-monopology.
If there's a way that the Web Both Auth standard might make their near-monopoly more harmful, we can talk about it, but let's focus on that -- the Web Both Auth standard itself is solving a problem that we in fact need solving, and seems to be designed properly for the use case. From my point of view as a site operator, it will actually help me allow in bot agents I want to allow in, that currently I'm being forced to block by trying to block all bot actors because of their expense to my site, without exception. I want to be able to make exceptions!
The giant wave of ridiculous distributed bot traffic of the past 1-2 years is very very real.