asciinema.org is currently being hugged to death! The btop stream is from one of the servers, currently reporting 95% CPU usage: https://asciinema.org/s/olesiD03BIFH6Yz1
But it's chugging along!
asciinema server is Elixir/Phoenix, runs on BEAM, and even with lots of connections and high CPU usage it serves request fine. Power of the BEAM. Phew.
Currently it runs on just 2 VMs, 2 GB of RAM each, so I will probably need to scale it up very soon anyway :)
In an age where everything is about clouds, it’s pretty amazing to see a high-profile service like this running smoothly on just a couple of 2 GB VMs. That’s less RAM than a mid-range laptop, and those machines struggle to run a modern browser.
Webservers require an insignificant amount of CPU time in comparison to any graphical interface.
Modern hardware in conjunction with smart software choices can be very capable, but it's pretty negligent to run a service people depend on with such limited resources.
Sure, not every site needs to run on a highly-available elastic cluster of nodes, but the alternative mindset of actively rejecting this architecture is arguably even more harmful.
A healthy deployment should be able to scale up and down easily, if not automatically. I wouldn't want to be woken up at 3am because prod1 needs a CPU/RAM/disk increase and a reboot, and the users wouldn't like that either.
There’s few reasons for this. First, I don’t have infinite budget for this, in fact the budget is close to 0 (sponsorships and donations help). Second, the servers are sponsored by Brightbox, and I’ve been running asciinema.org there for over a decade because BB is reliable and they’re great chaps, but I don’t have infinite resources available there. Finally, I don’t want devops complexity in my side projects.
> service people depend on
why wouldn't you cap a free service? imo it's more irresponsible to demand other people keep a thing you don't pay for running just because you happen to depend on it.
Just because a service doesn't require direct payment doesn't mean that its users shouldn't have any expectations of availability. There should be fair usage limits, of course, even if the service is commercial, but being caught by surprise by a traffic surge and boasting that it runs on underpowered hardware when the project is 10+ years old is irresponsible IMO.
Also, any service can have a financial purpose, even if it's "free" for users. This very forum is "free", yet the cost of hosting it is indirectly paid for by being an ad for the company that runs it. HN also famously runs on underpowered hardware that routinely fails to meet demand, but that is beside the point.
Feel free to NOT use asciinema.org if it doesn't meet your expectations for a reliable service. You can always self-host the server in your ultra-reliable, super-scalable K8S cluster.
Sounds preferable compared to be woken up at 5 am by your cloud providers billing alert.
Just scaled it up, seems pretty stable and responsive again.
That btop stream is lagging hard now, and jumping between full cpu and idle 0%.
Is the service crashing and restarting?
I was adding more VMs and restarting load balancer. Should be good now.
Long live the BEAM!
Tip: you might want to decrease the btop refresh rate. It's likely that at 100ms a considerable amount of CPU time is taken up by btop itself.
The CPU usage was single digit (%) with no viewers (but other requests being served) with btop at 100ms, so that's not adding a lot. But thanks!
Thanks for the great service! Is there a way to mirror the streams? Like a Wikipedia dump?