Just me or does this seems incredibly frightening to anyone else? Imagine printing a misaligned LLM this way and never being able to update the HW to run a different (aligned) model

It frightens me no more than the possibility of building a flawed airplane or a computer that overheats (looking at you, NVIDIA 12-pin) and "never being able to update the HW". Product recalls and redesigns exist for a reason.

If this happens, womp womp, recall the misaligned LLMs and learn from the mistake. It's part of running a hardware business as opposed to a software one.

I can't imagine they'd go for a full production run before at least testing a couple chips and finding issues.

The S in IoT is for security.