What if I’m a web alerts company? I crawl your content but my clients are all end users who actually see your content at your site.
What if I’m a web alerts company? I crawl your content but my clients are all end users who actually see your content at your site.
Fetch my RSS then.
The vast majority of news sites don't have an RSS. They don't even have a robots.txt.
Cloudflare and other WAF blocking is increasingly an issue with RSS feeds, especially if you use a centralized reader.
Then you are a uninvited bot that crawls my content in order to send alarms to people to tell them to visit my site?
This is a "what if it's for your own good?" argument. What if I break into your house to clean your toilets? Although amusingly, if you're a "web alerts company" that I didn't contract and sends out alerts in batches on your own schedule, you will probably send a ton of your customers to my site at the same time and slashdot me off of the web entirely.
No, I won't send a ton, I'll send a few dozens to a few hundreds at most because not everyone is interested in the same things. And they'll visit at their own timeline. If your site can't service a few dozen requests simultaneously then you probably aren't a news site in the first place so the whole argument is moot.