You are smart!
Another YC startup went this path, scraping targeting endpoints, and they didn't get much traction and they have since pivoted.
> We maintain a caching layer and avoid hammering websites.
This made me think of it. It is another way to avoid hammering websites by using extremely targeted requests for the most part bypassing HTML, DOM, and JavaScript.
100% yes, i found out about that company recently too, it's a tempting idea for sure
Why not much traction, was the benefit more to websites than businesses/scrapers?
it counts as "reverse engineering" a website which is much worse for the website owner imo, then again im biased