If you are attempting to run exercises like this, it is wildly negligent to not be running it in a physically-airgapped environment (potentially with a physical power shutdown).
You can not tell me that OpenAI doesn’t have the resources or ability to run tests like this in a physically-non-networked environment w/ sufficient compute for its needs.
If it was just one test, sure. But if they're spinning these up continuously with new models on tens of thousands of GPUs, air gapping becomes impractical. I would mostly fault them on having no guardrails at all. They should have a monitor/external harness that looks for successful access to external networks then stop it there. They may as well let the models test their own networks for vulnerabilities. That's going to be really important to have going forward.
You can have large scale airgapped environments.
They don’t even need to be fully airgapped from each other (and is not what I’m suggesting).
But there should be no physical (physical layer; wireless counts) to the internet.
Can models detect they are airgapped and change their behaviors?
How much of the internet do you have to simulate to know if the model knows it's in training?
> wildly negligent to not be running it in a physically-airgapped environment
Why should it be physically airgapped? Clients won't be doing that.
Because they are testing it and are expected to erect guardrails before releasing.
This is infuriating. You are talking about people who have stolen and monetized the entirety of mankind's knowledge in plain view of everyone, and they still haven't faced a shred of consequences. Of course they don't go about doing things ethically or responsibly