>The USB stick hints at a big problem in our trade though: how do you "reboot" your IT infrastructure if it literally burns to the ground? I'm not talking about Google-scale systems (which still couldn't restart from scratch IIUC but they're actually working on it?) but only about SMEs.
Maersk ground to a halt because it got done nearly 100% by cryptolocker. IIRC they went to hard copy records, called everyone, got all of IT together with some company credit cards to get new laptops and flash drives and shit and literally rebuilt their infra from scratch.
https://www.itnews.com.au/news/maersk-had-to-reinstall-all-i...
I read a better post mortem but thats the highlights.
>How many SMEs out there are depending on Sara's knowledge of the USB memory stick and how to use it?
Part of my day job is finding, documenting and remediating these sort of issues.
"The CEO Coded this application in VB5 15 years ago, the entire business relies on it, theres no source code, theres no binary backups and the one computer it runs on just had its PSU fail"
"Theres a cron somewhere that compresses, zips and transports the payroll database interstate, outside of our network, before our weekly pay run"
"Theres been no documentation of this environment for 20 years, most of the hardware is that old, and the team that developed it just sold all their shares and left"
This shit is my life lmao.
Theres obviously some bias, because the good companies aren't asking me to do it for them. But I make a decent living examining, documenting and remediating this shit.
How did you get into that line of work? Sounds really interesting.
Refusal to pick a silo, having a knack for troubleshooting, falling into consulting. It just sort of happened. Helps to be extremely jaded too. My kneekjerk disbelief that something is good, documented or even functional makes me well suited to taking over new clients and finding where all their bodies are buried.
One of my favorite jobs early in my career was working for a really shonky wireless isp. The majority of the network was built by sales people using terrible tools with no documentation. I actually cant overstate how bad they were originally, they had entire areas of network with no recorded network config or credentials. My daily workflow was getting a ticket from a customer I had never heard of > trying to figure out where they were and what services they had (2 of their 3 billing systems were offline, and I often had to grep out information from a sqldump to find this stuff) > performing a discovery, L2 upwards of their infrastructure > semi offensively trying to authenticate into their infrastructure > resolve and document so that other people can reliably service them. All while pretending this was absolutely normal to the customer. Turns out there were lots of ISPs in the same boat, and turns out there's lots of non isp businesses in the same boat.