> The way they store data in a key/value system really hurts the performance
It doesnt, unless your site has a lot of post/product/whatever entries in the db and you are having your users search from among them with multiple criteria at the same time. Only then does it cause many self-joins to happen and creates performance concerns. Otherwise the key-value setup is very fast when it comes to just pulling key+value pairs for a given post/content.
Today Wordpress is able to easily do 50 req/sec cached (locally) on $5/month hosting with PHP 8+. It can easily do 10 req/sec uncached for logged in users, with absolutely no form of caching. (though you would generally use an object cache, pushing it much higher).
White House is on Wordpress. NASA is on Wordpress. Techcrunch, CNN, Reuters and a lot more.
Just want to point out that your 50 req/sec cached means nothing in case of dealing with scrapers. What is the entire topic ...
he issue is that scrapers hit so many pages, that you can never cache everything.
If you website is a 5 page blog, that has no build up archive of past posts, sure... Scrapers are not going to hurt because they keep hitting the cached pages and resetting the invalidation.
But for everybody else, getting hit on uncached pages, results in heavy DB loads, and kills your performance.
Scrapers do not care about your top (cached) pages, especially aggressive ones that just rescrape non-stop.
> It doesnt, unless your site has a lot of post/product/whatever entries in the db
Exactly what is being hit by scrapers...
> White House is on Wordpress. NASA is on Wordpress. Techcrunch, CNN, Reuters and a lot more.
Again not the point. They can throw resources onto the problem, and cache tons of data with 512GB/1TB wordpress/DB servers. By that, turns WP into a mostly static site.
Its everybody else that feels the burn (see article, see the previous poster and other).
Do you understand the issue now? WP is not equipped to deal with this type of traffic as its not normal human traffic. WP is not designed to handle this, it barely handles normal traffic without throwing a lot of resources on it.
There is a reason why the reddit/Slashdot effect exists. Just a few 1000 people going to a blog tend to make a lot of WP websites unresponsive. And that is with the ability to cache those pages!
Now imagine somebody like me, that lets a scraper lose on your WP website. I can scrap 10.000 pages / sec on a 4 bucks VPS. But each page that i hit that is not in your cache, will make your DB scream even more, because of how WP works. So what are you going to do with your 50 req/s cached, when my next 9.950 req/s hit all your non-cached pages?! You get the point?
And fyi: 10.000r/s on your cached pages will make your wp install also unresponsive. The scraper resource usage vs WP is a fight nobody wins.