I am an AI Safety Person and I want the government to nationalize or have a significant stake in the frontier labs and to have democratic control of the development of the technology. The AI Safety movement is not a monolith. I do not think Eliezer Yudkowsky nor his acolytes should hold the reins, but a lot of folks sure like to create a strawman that anyone who wants to regulate OpenAI is somehow an EA/MIRI weirdo
>but a lot of folks sure like to create a strawman that anyone who wants to regulate OpenAI is somehow an EA/MIRI weirdo
Anytime I see a random person on X who makes these kind of safety alarmism posts >90% of the time can be directly tied back to EA / LessWrong / related offshoots. You can not deny the amount of people, employees of Antrhopic / OpenAI, CEO and employees of various AI companies, etc are related to these groups.
Yes I trust this government with that power. /extreme sarcasm
Yes I trust sama/dario/elon with that power
/even more extreme sarcasm than you
I don't trust them, which is why I inherently question the validity of their RSI/bioweapon/extinction claims.
You trust trump more than any of those three? Really?
You can have an arbitrarily low opinion of Trump and still trust in democracy as a form of government over autocracy or oligarchy, which is what the AI labs have.
That's an extremely fair objection! But despite the many, many flaws of our government and the current administration, I still have (hopefully) a chance to vote them out of power. I have no such hopes with Sam Altman or Elon Musk.
> and to have democratic control of the development of the technology
Why? I don't care about democratically participating in a closed model's development. It doesn't belong to me.
China will develop whatever they want, a federal stake in OpenAI or Anthropic punishes Americans and shields US labs from legitimate competition.
In my mind, democratic control of the technology means that we (the govt, or other empowered agency) take ownership of their assets and IP, solve or find a level of alignment or guardrails that society is comfortable with. Then we distribute the technology, or access to it, to avoid power concentration. This would also certainly require international coordination with China on a slowdown or pause, which I think is possible.
> This would also certainly require international coordination with China on a slowdown or pause
Personally I think the country with a strictly meritocratic elite selection system that also just outright kills you if you sell weed will have a hard time sympathizing with Bay Area thinkers who talk about AI killing us all during their ayahuasca breakfast before returning to their meth fueled crunch towards releasing the next version of the AI that will kill us all.
>I think the country with a strictly meritocratic elite selection system
Not really relevant to the broader discussion, but this simply isn’t an accurate description of China. Starting with the gaokao, admission quotas are set by province and admits to Peking university and Tsinghua are disproportionately from the urban professional class. Candidate party members must be politically vetted, which means that people whose families have expressed anti-communist views, are members of banned organizations (e.g. falun gong), or have substantial criminal records will not be permitted to advance. And once you make it into the party and enter political service, your advancement relies upon opaque patronage networks that someone without connections is unlikely to be able to navigate, even if they successfully satisfy the economic metrics the state assigns them.
I don’t want to overstate this, the Chinese system does filter out a lot of chaff and the current Chinese leadership has a lot of very capable people in positions of power. But I do not think it is substantially more meritocratic than Western political institutions
> substantially more meritocratic
It is substantially more meritocratic on domains that matter for governance, your analysis of the incentive structure between systems is off.
1) this 2026, old school CCP patronage networks are broadly dismantled.
2) even in the mass patronage, mass corruption days, system selects for BOTH corruption competence AND performance competence for the simple reason a CCP bureaucrat has to start from the bottom and climb up, which means they need to be good with patronage AND they need to be good with hitting development KPIs. More meritocratic they are at doing their jobs, the higher they climbed, the more they get promoted and more $$$ to graft, because ability to graft directly tied to actual job competence. Hence even cliques/patronage network has to select for actual competence. This works in PRC because there are many people, and hence pool of competence is high, they can have BOTH corruption and competence, i.e. whatever pool they draw from is ultimately filtered by performance meritocracy due to incentive structure. There is reason why PRC only country where positive corruption levels was correlated to positive growth.
This is not the western system where any idiot can enter politics at anytime, and they only domain they need to optimize for is popularity to get votes.
CCP cadre evaluation strictly does not evaluate on popularity domain. It focuses on administration/execution and in so much it needs to focus on patronage... which btw any political system has (factions/cliques)... the patronage system still selects for execution, not popularity. On side, functionally what west politics selects for IS mass patronage (popularity), so attention meritocracy and not performance meritocracy, aka completely stupid incentive structure for governance. West also has ample, ample corruption, "legalized" under lobbying and paper pushing industries, so I suppose west also meritocratically selects, except for lawyers etc, and KPIs is # of document generated and not # of things build. The two are not the same when it comes to nation building.
You're in luck, neither I nor anyone else want those people near a negotiation table.
Yikes. Hopefully you never gain any power or influence. The last thing we need is more economic central planning. I sincerely hope that you fail.
Do you think things like the Manhattan Project were a mistake? Comparing AI to nuclear weapons is perhaps a stretch, but I think most people recognize that certain technologies or artifacts are best monopolized by our governing bodies. I think if the capabilities of AI systems keep growing on trend, it is not unreasonable to think wonton usage could disrupt society or cause mass harm.
to have democratic control of the development of the technology
Not super interested in what the people who elected Donald Trump POTUS twice think about AI.
(Of course, with Musk and Brockman in the C-suites at two of three major labs, that's what we'll get either way.)
Fair objection, but I still think it is lower risk to diffuse power and control of a potentially dangerous and revolutionary technology than to leave it in the hands of the few elites who have not show much ethical integrity so far.
Is Greg Brockman right wing?
He's the largest single private donor to the Trump 2024 campaign, as I understand it, at $25 million.
Interesting, I did not know this.