OK, I have a fun one. I was a rookie back then, it was 2007, and the cloud was only beginning to take off. My company had a physical server room.

Automated CI was not quite as big back then as it is now; we had some in-house CI-ish facilities, one of which was buildbot: a small Python script that continuously built all of our projects one after another, reporting successes/failures in a simple UI.

One day I was puzzled to discover that buildbot hung trying to build a tiny Java applet. Just a handful of .java files. But it had been stuck for more than four hours at that point.

I ssh’d into buildbot and looked at what exactly the hung process was doing. It was not javac. It was jarsigner.

Then it dawned on me. jarsigner needed some crypto-safe randomness, it was trying to get it by default from /dev/random, not from /dev/urandom, and the machine ran out of entropy!

I ticketed a change, but as an interim measure, I went to the server room, switched the console to buildbot’s server, and moved the mouse vigorously for some seconds. It proceeded.