OP point out that OpenAI used the "too dangerous to release" marketing ploy with GPT-2... Positioning this as "both sides" have played this card.
But at this time Dario was at OpenAI and was a co-author on the GPT-2 research paper announcing the model.
The "too dangerous to release" approach has been him the whole time, at both companies.
I said this in the other thread, but they were proven right about their gpt2 worries, weren't they?
From the original 2019 release:
> We can also imagine the application of these models for malicious purposes , including the following (or other applications we can’t yet anticipate):
> These findings, combined with earlier results on synthetic imagery, audio, and video, imply that technologies are reducing the cost of generating fake content and waging disinformation campaigns. The public at large will need to become more skeptical of text they find online, just as the “deep fakes (opens in a new window)” phenomenon calls for more skepticism about images.These worries are why they stated they were cautious in rolling it out
https://openai.com/index/better-language-models/
> The public at large will need to
Ah, yes. You see, it’s not them who are wrong for knowingly releasing something they knew to be harmful, it’s everyone else who needs to change. That seems reasonable. Humanity is famous for being able to rapidly adapt to fast changes as one voice. Oh, wait…
They are no different to the tobacco and oil companies. They know the harm they’re causing but care about personal profit about everything else.
I'm not an AI booster, but in this case I'd say that pausing the rollout for mitigations (such as public education) to be put in place was the responsible course of action.
With the benefit of hindsight, you can certainly argue that the pause wasn't long enough or that the mitigations weren't sufficient. But that wasn't a view held by many at the time - indeed, it was mocked as a marketing ploy (and still is; see gp's post as evidence).
> pausing the rollout for mitigations
What mitigations? Nothing they’ve done is relevant to the four points in the comment above.
> such as public education
Their “public education” is about as meaningful as alcohol warnings.
https://www.youtube.com/watch?v=Xj4aRhHJOWU
> With the benefit of hindsight
No hindsight needed. These problems were obvious from the start. Not just to me but to many others. Clearly also to them.
> indeed, it was mocked as a marketing ploy (and still is; see gp's post as evidence)
Two things can be true at once. Of course it’s marketing to say “this is too dangerous to release” if they’re going to do it anyway. Either that or they’re so supremely irresponsible and greedy that they don’t care about the consequences as long as they can profit. And again, all of those can be true at once.
Also, worth noting that when they talk about it being “too dangerous”, they’re usually talking about fantasy scenarios of the AI gaining sentience and enslaving humanity. But there are many other dangers (as listed in the comment above) to consider that come from humans directly misusing the technology.
> What mitigations?
They did try to place limits on their API, and tried to develop classifiers for AI-vs-non-AI text (which was abandoned in 2023, in a world of many models). A lot of their efforts in those days seemed to be to work with Universities to figure out what to do about all of this incoming tech. They weren't the first to develop a language model.
> when they talk about it being “too dangerous”, they’re usually talking about fantasy scenarios of the AI gaining sentience
They didn't talk about "it" (that model) in those terms, as mentioned above. Or the following few from what I can see. They seem pretty specific about each model's risks and publish what they can find in the model card. But yes, they have a fear of where things may be in the future if models keep progressing.
I don't personally think talk of it being "too dangerous" is good marketing if the goal is to get rich. It invites restrictions from governments and others. I don't know anyone that picked a model because it was apparently restricted: most of their funding comes from Companies that are generally risk-averse. Online AI hype seems to mostly come from the demos, not the doomerism.
I do think there's an uncomfortable trade-off involved in all of this, and some of it comes down to whether you think the tech will be developed regardless of your participation. I believe the people in labs like Anthropic are worried yet think they are better off steering it the right direction, so they push on.
Yes, it's not their fault, that people are using the tool they made in a malicious way.
I hate ClosedAI as much as the next guy, but this is an extremely illiberal take. It's not the kitchen knife manufacturer's fault that people are using their product for murder, it's not my fault that people are doing crimes over the Tor relay I run.
The Tobacco industry is evil because it misleads the public about its product being poisonous and bribes politicians through widespread corruption. Tobacco is also different because it is not a neutral tool that can be used for good and bad, but poisonous and will harm you no matter how you use it.
> Yes, it's not their fault, that people are using the tool they made in a malicious way.
Yeah! It’s not like they predicted these malicious uses before releasing the tools. And it’s not like they’re making them available to a dysfunctional government in order for them to militarise the technology and… Oh, wait…
> but this is an extremely illiberal take.
I’d appreciate if we stopped this Americanised version of poisonous discourse where everything is reduced to a box in a vague political ideology. By this I don’t mean politics don’t matter—they do—but not everything is black and white, right and left, or needs to be categorised to be discussed.
> It's not the kitchen knife manufacturer's fault that people are using their product for murder, it's not my fault that people are doing crimes over the Tor relay I run.
Always with the kitchen knife. That’s not an argument, it’s a talking point. Explosives are tools too, as are machine guns. No tool is entirely neutral. LLMs are not comparable to kitchen knives. Death is not the only possible bad outcome.
> Tobacco is also different because it is not a neutral tool that can be used for good and bad, but poisonous and will harm you no matter how you use it.
Tobacco is not just cigarettes.
https://leafngrainsociety.com/featured/10-surprising-uses-of...
Maybe because Dario was actually reasoning through potential risks, rather than blindly thinking everything will be okay?
Thinking GPT2 was too dangerous is and was absurd.
It's dramatic for sure, but at the time it was genuinely alarming to think through the implications of a machine being able to generate plausibly human-authored text. I think many of the alarming implications have in fact come to pass, and the world is a more strange and dangerous place than it was before.
At that time, nobody believed a dead internet was technically feasible. Maybe this is hard to remember now.
The "danger" was in terms of spam / misinformation proliferation, not the same category of capabilities adjacent risks current discussed.
You can hold your own opinions on spam/misinformation as a problem, but to say there was no credibly anticipated outsized downside to a sudden jump in human-passing text generation feels pretty off to me.
I remember the arguments back then. Those alarmists were wrong. Nothing happened or could've happened just because you could generate drunken ramblings.
It's the kind of people that want to ban anything because of some theoretical small harm is technically possible. We're lucky it's not more prevalent or we'd still be in the stone age.
It wasn't just "drunken ramblings," they were right about the dangers they called out. Reddit is largely LLMs arguing with each other, it's so easy and only costs a few thousand dollars to spin up a mass misinformation campaign.
They were right about the risks.
It's amazing to me that people actually thought GPT2 produced "human-passing" text, while I'm still tripping over obvious LLMisms in the output of recent models on a daily basis.
(It's also amazing to me that it took mere minutes for this observation, deep in a sub-thread, to get downvoted without any reply, with no obvious reason for it.)
People may perceive you to be cheaply mischaracterizing the argument.
Nobody believed or suggested that GPT2 could do longform or produce novel text that stood up against careful scrutiny as insightful or well informed. But because the capabilities were novel, people would have no strong alternative than to believe some person wrote it.
You current tripping over LLMisms is irrelevant. You have years of antibodies, both personal and herd-immunity (eg, the many, many articles and comments that describe LLMisms).
LLM-isms are much less prevalent in base models, which is what GPT-2 was. It had significant problems with maintaining coherence, but GPT-2 generated text did not have the obvious tells of today's LLMs.
All that happened was announcing said potential risks and then continuing to launch and push and release it anyways. Something that Anthropic continues to do under his leadership.
Google also analyzed for risks earlier in development of "Meena" and "Bard" and chose not to release, instead. And then got caught flatfooted when OpenAI did so anyways. (They also I think didn't really see a compelling business case for public propagation of it, either).
It starts to look very much like what is really happening with Anthropic is a lot of cynical attempts at regulatory capture. Make grand proclamations. Get your stuff out there first. Ask for regulation and then kick the ladder away from underneath you. But they did it clumsily and failed to grease the right palms.
To be fair, generative AI is wrecking society in new and unexpected ways every week. From lies and misinformation to people choosing chatbots over therapy and relationships, there's a profound impact on society that will only get worse in the coming years. The look for junior programmers who are capable enough to get anything done when the AI is down has been depressing, and things are looking much worse for the years to come.
Important safety features ("do not generate child porn", "statements should be factual or backed by evidence") were simply not part of the design of these systems and have yet to truly solved to this day, but AI companies decided to release these technologies onto the general public regardless of their glaring flaws.
I like AI for its shitposting capabilities and its neat parlour tricks, but I also believe so far it has been a net negative for everyone but the richest minority of society who benefits from firing people and having computers do half their jobs badly. It's too late now, but in hindsight I do agree that these systems were too dangerous to release in this shape.
I feel like it could be a law that there is essentially no way to guarantee that AI is any more or less safe than humans. It kinda seems incompatible with what we understand to be "intelligence" which arguably requires a certain unpredictable freedom...Has a method of "baking in" such safety features even been conceptualized? Or is it just a matter of nurturing/raising/policing them after the fact and hoping for the best like with us?
Kind of taking this a step further, what is the worst thing that an AI run amok could do to existentially threaten the human race? I feel like almost anything could be short-circuited by some form of "pulling the plug" before it got too far. But, hypothetically, if it were possible to launch nukes without human intervention, or with maybe a small amount that could be socially engineered, that seems plausible (or releasing some kind of super-pathogen that is stored in a lab somewhere).
So, what if, along the lines of MAD doctrine and the plot of Battlestar Galactica, the best thing we could do for AI safety is just to engineer our other systems so that a hypothetical superhuman adversary could not use them against us? Which is just making our world safer all around rather than trying to kludge arbitrary limitations into an "intelligent" system.
(This doesn't really solve AI child porn and fake news but those things are mostly just imaginative reflections of the people using them and you can't really fix that any more than you can stop people from doing it themselves)
>Kind of taking this a step further, what is the worst thing that an AI run amok could do to existentially threaten the human race?
Probably convincing humans that untrue things are true. Think less "AI launches nukes" and more "AI convinces people in country A that country B is trying to subvert & destroy them, while it does the same for country B wrt A."
Social Media has been used for Balkanization of the US for at least a decade. It's how Russia said they would weaken the US, and exactly how they have. Now Social Media is being used with LLMs to make it even more effective. Social media is a blight. The algorithms should be basic enough for an average person to understand (chronological, friends only, etc). Also, f the marketing that encouraged manipulative algorithms.
AI is often better than therapy as reported by users. Therapy has some inherent dark-patterns that AI doesn't have yet, like the therapist's financial incentive to trickle a solution to your problem to preserve their income.
AI gives more than a billion people instant access to knowledge, it is starting to accelerate scientific research, it democratized software development, design, and illustration.
I strongly disagree with the opinion that it has been a net negative.
Lies and misinformation, or choosing chatbots over therapy and relationships might sound scary, but as of now I see practically negligible impact there. Even social media is still roughly as usable as before AI.
With ai I can get government bodies to do their job. Escalate and make formal complaints until it’s done. Manually would take ages to find the right law and draft a proper complaint.
Dario's brain child