> I personally think it is inevitable that China and the US stops producing frontier level open models when they become too valuable.
This reads like a paradox, though. If the value of a FOSS model comes from pushing the local frontier, then FOSS releases will be even more attractive in a world with highly capable AI services. The underdog researchers will be motivated to train smaller LLMs that close the gap with the most newest architectures and improvements, democratizing those same benefits to local models. There's not really anything that stops them, besides finding an investor.
The US already has top secret AI technology like the NRO's Sentient. The danger is less the architecture, and more the data that it has access to and the technology that it interfaces with. Most people aren't entirely convinced that AI will ever be "too valuable" in the first place.
> Most people aren't entirely convinced that AI will ever be "too valuable" in the first place.
There will be a generational leap that makes current LLMs obsolete in every way. Let's say China gets there first, why would the CCP let it be released publicly? That is an insanely valuable advantage in everything from war fighting to economics and more.
Humans for millennia have used technological advances to make better weapons. AI will be no different, unfortunately. Wars will be fought with drones and artificial intelligence from now on.
> Let's say China gets there first, why would the CCP let it be released publicly? That is an insanely valuable advantage in everything from war fighting to economics and more.
Because China is based in a completely different set of values than the USA. They look at the world differently, and have a different understanding of it.
Your assumption that your attitude is pure human nature is deeply grounded in your culture, which (assuming you are from the USA) is deeply grounded in personal competition rather than co-operation. All that rugged individualism. Other cultures are less based around that, and don't see the world as a zero-sum competition for survival.
I'm no expert in Chinese culture, but there are folks from China on this topic who could probably answer this from their perspective.
Culture does not exist in a vacuum -- it's downstream of fundamental human motivations that are manifest in all times and places.
And using "culture" as a general concept to dismiss concerns about specific behavior, without actually attempting to posit any particular cultural values, or describe what mechanism actually causes those values to have a determinant effect on behavior, is equivalent to saying nothing at all.
Whatever official party line the current Chinese government espouses, it's plenty clear from even a cursory understanding of Chinese history, from the Warring States period to the warlord-era conflicts of the early 20th century, to the current behavior of the CCP, that China does not have any effective cultural guardrail against factions seeking advantage over opposing factions, which is a phenomenon that has been present in all societies that have ever existed in human history.
This doesn't pass the sniff test.
Consider that the US was intimately involved with the start of the FOSS movement. Compilers and commercial operating systems are hugely valuable.
Also consider that history the world over is littered with counterexamples. Historical China itself is rife with examples of technological secrecy and exclusivity.
No competent government will voluntarily release (or allow to be released by its subjects) technology that would give them a meaningful advantage on the world stage.
"I'm no expert in Chinese culture"
You've made that plenty clear.
Are you an expert?
You're making an assumption that's debatable, but let's just assume it's correct. The reality is that China won't be the only to achieve such. In the same way that the US put arguably the best minds in the world towards developing the atom bomb, and within a few years the Soviets had theirs as well. Shortly thereafter they were popping up all over Europe. It's kind of the story of LLMs in general - once people saw it was genuinely possible, we went from one LLM company to them showing up everywhere.
It's highly improbable that any country retains a meaningfully persistent technological lead, and the soft power to be gained from providing models cheaply for society will probably greatly outweigh any sort of temporary strategic advantage that 'hoarding' could provide. And the reality is that nuclear weapons remain the kings of war, by an overwhelmingly wide margin. If the US and China or Russia got into a "real" war, that's pretty much the end of the world.
> You're making an assumption that's debatable, but let's just assume it's correct. The reality is that China won't be the only to achieve such. In the same way that the US put arguably the best minds in the world towards developing the atom bomb, and within a few years the Soviets had theirs as well.
I never said other countries won't gain access to the technology. The point is that the 1st country to achieve that leap will get to use it to gain advantages in literally every sector.
I get some people are hoping that AI research is for the benefit of all, and I do too, but that is wishful thinking.
> The point is that the 1st country to achieve that leap will get to use it to gain advantages in literally every sector.
That's still speculative. We have something of a basis for speculation with the way LLMs influenced software engineering, but it's not necessarily applicable to "literally every sector" of scientific progress. If their quantum leap is anything like vibe coding, then it will quickly become an unmanageable cesspit lacking accountability and grasping at notoriety. We have precious little to show for half a decade of LLM-generated code, nobody disrupted the FAANG incumbents by using an LLM to crack the digital secrets of the universe.
The safety angle played its card and lost. We've been generating trillions upon trillions of tokens, and nobody has raised the dead or made The Bomb with red mercury. Smarter AI clearly isn't a panacea.
> I get some people are hoping that AI research is for the benefit of all, and I do too, but that is wishful thinking.
You're way too lost in the plot. Technology can absolutely be used for ill, and no doubt AI has already contributed to reprehensible things. But it cannot turn lead into gold. It cannot reveal the face of God or compute The Answer to the Ultimate Question of Life, the Universe and Everything. The ultimate threat of AI is the stuff it can already do - cosine similarity search on billions of discrete records, automated labeling with computer vision, multimodal analysis and corroboration pipelines. The hypothetical scenario where superintelligence is made is a thought-terminating cliche that is driving people into a zealous fervor.
Yes I am speculating a leap past LLMs. They are on the edge of what I would actually consider AI.
I believe it is an inevitability and I'm not speculating whether it's in 10 or 50 years.