It is deliberately targeting known AI-scapers, or anything it sees as a bot?
Could it be that other scrapers are pulling the pages, without making the extra requests to get ad related resources, to present the content to people ad-free, and embedding the ads in the main response body is a way to get around that so the human sees an advert at least, even if it isn't the one they might see if the stalky-adtech-algo could deliver something more targetted.
Google itself now captures a lot of traffic with LLM generated site summaries.
For media/news businesses that make their money from readership, it is unsustainable to subsidize Alphabets content farm. Thus, understandable people would change their corporate posture with a search turned scraper company. =3
This is why Cloudflare banned Googlebot. Googlebot is no longer a search engine bot and now an AI scraping bot.
No, Cloudflare banned Googlebot because they want the racket money. Scraping edge is a pure wordplay to convince you their racket is actually a good thing!
Cloudflare is famous for IT service rug pulls to get paid back for high traffic loads.
Let us not forget they are a business =3