There are a lot of options in this space, but I feel utterly helpless. Which of these are really secure such that I could intentionally run malware without fear of my ssh keys being stolen?
All of the options I find seem to say something along the lines of, “We’re totally secure so long as you don’t Fizz the Bar or Twiddle the Quanzipulator”.
What is the most bullet proof, idiot proof option I should use?
An interesting book "Countdown to Zero Day: Stuxnet and the Launch of the World's First Digital Weapon" describes how the researchers at Symantec who analyzed Stuxnet ran malware:
"They worked in Symantec’s Threat Intelligence Lab in Culver City, the cyber equivalent of a biodefense lab, where researchers could unleash malevolent code on a “red” network—a sandboxed system air-gapped from Symantec’s business network—to observe its hostile behavior in a controlled environment. To reach the ground-floor lab, workers passed through several sets of security doors, each with progressively more restrictive rules. The final gateway kept all but a handful of workers out and physically isolated the red network from computers connected to the outside internet. Portable media were prohibited here—no DVDs, CD-ROMs, or USB flash drives were allowed —to prevent workers from mindlessly slipping one into an infested machine and inadvertently carrying it out of the lab with a malicious specimen stowed away on it."
A VM that doesn't contain your keys.