I don't think it's a good idea to give the crowds that kind of weapon. The first thing they'd do is "liberate" the model aka remove guardrails and safetly-protocols and brag on X / reddit with it and throw it into the public. That's only cool for a geek that doesn't think about the ethical impact of such a move. You'd basically become responsible for anything that is done with it, forever - have a good sleep. /s
As opposed to what, the US military, or better yet Israel (because we all know they won't be excluded) using that model to drive weaponry that kills people?
Your hypothetical implies that there is a better alternative, but when those models are "restricted", in practice that means that the only people who have access to them are precisely those who can and will use them for the worst kind of shit. So yes, releasing them to the public is a better deal, ethically speaking, at least then the playing field will be slightly more equal.
There are plenty of weapons (see custom made virus) which no state actor (or even an informal militia) would want to release, as these weapons attack everyone. But, open access to details of its construction leaves everyone vulnerable to motivations of small groups of crazy individuals.
What if I told you there are no safety guardrails. I used GLM 5.1 and had fable literally build a harness to avoid triggering guard rails. I built skills carefully and had Fable doing vuln research and exploit repro in a few hours. I called the project manhattan. The GLM models are down for almost anything so I named it Oppenheimer. It orchestrated the fable CLI agents via tmux. This whole Fable/Mythos thing is such a fucking joke. It is all PR and theatre and they know it.
I’ve been doing pentesting with LLMs for a while and only hit a few “nope I won’t do that” and one “this conversation is flagged for being against the TOS”. No idea what the guardrails are but they are trivially abused