And if you whack someone with a wrench until they tell you the password, it's even easier!

Seriously, if someone is getting physical access to the machine to the extent where they can remove the hard drive... I doubt that it makes a difference whether the browser's password manager keeps its passwords encrypted in-memory.