I think if you are worried about AI killing us and the like the main protection is the physical world - you can pull the plug. Data centers don't run fast. Alignment is all very well but there will always be bad humans to tell it to do bad stuff which is why you get malware in traditional software.