In particular: *all the knowledge that AI has of nuclear weapons is freely available on the internet*. It's not superhuman, and there's no secret sauce data. If you just study the same PDFs and blog posts it has, you will acquire the same abilities. I cannot imagine anyone with the intent and immense financial and political resources to actually build a weapon would say that some study time is the only thing stopping them from detonating a nuke.
It is pretty convenient for the labs to frame the conversation around this though, since it is easy to address, very few paying customers are rejected, and sounds scary (so surely the less scary sounding stuff must be solved right?)
My hypothesis is that making the knowledge of how this stuff works accessible to the public results in a lot of false-positives (from people just playing around) that intelligence agencies have to then sift through / tune filters against; which creates a noise floor for real foreign nuke programs to hide in.
So governments ban anything that could result in false positives (since nobody needs to be doing any of that stuff outside of designated labs anyway), to lower that noise floor; to in turn make catching the foreign nuke programs tractable.
(It's a bit like how fancy mansions always have a completely flat and barren part of the property between an outer perimeter and the start of any gardens/outbuildings/water features/etc. That barren area is a killbox: since nothing is supposed to be there, anything at all that does appear there is a valid target for the manion's guards to shoot at [or otherwise engage with], without needing to get a clear identification and command approval first. This wouldn't work if the killbox was covered in vision-obscuring decorative features; nor if the mansion had employees, animals, etc. that had a valid reason to wander into the killbox. So such things are prevented, in order to make the problem of perimeter security tractable.)
But this knowledge is readily accessible today. At least for manhattan-project level bombs. For later developments you mostly get simplified overviews with important details left out. But even there you have communities speculating about this very publicly
The same is true for adjacent topics. Most LLMs will refuse to tell you how to make dynamite, youtube demonetises any videos about it, but it's right there in the wikipedia articles on dynamite and nitroglycerine
Administrative convenience is no excuse to limit individual liberty, capacity, or knowledge. Individuals come before states!
Usually measures like these aren’t to stop the people with those kinds of deep resources.
With everything, there is a much bigger group of people in the middle that have “some resources” and “some desire” that these measures are surprisingly effective against.
Raise a $20 item by $1 and suddenly there’s fewer interested people, even though the cost difference is minor. Well, minor to some people but not to others.
But is limiting this information in an LLM the right move? Well that’s a different question.
The difficulty with creating nuclear weapons has been 99% in refining and processing the fuel, not the structure of them, for a very long time.
True for fission bombs. Less true for fusion bombs. The principal makeup and manufacturing of fusion device parts like tampers are still unknown to the public. Having a supply of HEU does not tell you how to assemble a functional triple stage device or how to utilize tritium, an isotope that measurably decreases in purity by the day.
You need a fission bomb to ignite a fusion bomb btw.
[flagged]
Was something I said unclear?
The purpose was to clarify that the obstacles to constructing modern nuclear weapons is not accurately characterized as "99%" fuel-related. Even if a group were to obtain a stockpile of ready HEU and plutonium-239, there is knowledge they simply will not have because they did not spend a trillion USD testing different bomb configurations last century. The difference in yield is two orders of magnitude.
And that is relevant how?
I'm sure something in the few dozen kilotons range doesn't need all that stuff,
while still giving you more than enough heat-rays to "enjoy".
Aka this has zero relevance to the proliferation discussion. Anyone having the problem you are describing long ago already created a basic nuclear stockpile.
Notably, neither China nor Russia seemed to have issues creating Thermonuclear weapons despite the shortcomings you identified either.
seriously, what was the point of this comment?
It's a way for AI labs to discuss safety while misdirecting from more mundane but widespread harms such as spam.
its also hilarious when you consider that building nuclear weapons is fundamentally a supply chain problem. The taliban isn't going to suddenly have nuclear capabilities by asking chatgpt. Any adversarial nation that has the means to extract and concentrate fissile nuclera material probably has HUMAN scientists who spent years studying the problem in well funded labs.
That's rather meaningless. The scientists in the Manhattan project initially had less information than what is now available on the internet.
> The scientists in the Manhattan project initially had less information than what is now available on the internet.
On the other hand: the Manhattan project had access to much better physicists than the typical terrorist group has. :-)
But physicists today have much more information and compute and could be more productive.