An api rate limit router - it’s gives a queue per user to solve noisy neighbors, it allows the user user to reroute 500s to other regions automatically, the api provider can tune the speed of request with headers. It should be a massive speed boost because non-beam servers waste performance on sleeping threads. It’s works by my edge servers doing all the retrying across different regions and it can send webhook to you across regions which means we delivery if you an all your downstream are up in one region. It’s actually already ready but of course no one wants to trust me with their traffic. Nobody wants to even read the design doc nor sdk. I’m hoping that upcoming new agents bring so much traffic that people look for this category of software. Until then I guess I am Cloudfare before the world got hit by bot scrapers.

The modern orchestrator are reactive, they don’t handle spikey traffic. Your favorite retry library will cause retry storms for downstream dependencies and your public APIs. Remember EZThrottle blog posts

EZThrottle.network