Dude - I can't believe we're at the point where we're publishing headlines based on someone's experience writing prompts with no deeper analysis whatsoever.
What are the exact prompts and sampling parameters?
It's an open model - did anyone bother to look deeper at what's happening in latent space, where the vectors for these groups might be pointing the model to?
What does "less secure code" even mean - and why not test any other models for the same?
"AI said a thing when prompted!" is such lazy reporting IMO. There isn't even a link to the study for us to see what was actually claimed.
Agreed but tools that allowed lay people to look at "what's happening in latent space" would be really cool and at least allow people not writing a journal article to get a better sense of what these models are doing.
Right now, I don't know where a journalist would even begin.
I don't think even the people at the forefront of AI are able to decode what's going on in the latent space, much less the average joe. We are given these clean examples as illustrative, but the reality is a totally jumbled incoherent mess.
Not true at all. You can take a vector for a given embedding and compare it to other things in that area of latent space to get a sense for how it is categorized by the model. You can even do this layer by layer to see how the model evolves its understanding.
That was pointed at Crowdstrike - the authors of the study - who should definitely have that skill level.
“Any sufficiently advanced technology is indistinguishable from magic.”
The average- nay, even the more above average journalist will never go far enough to discern how what we are seeing actually works at the level needed to accurately report on it. It has been this was with the technology of humans for some time now - since roughly the era of an Intel 386, we surpassed the ability for any human being to accurately understand and report on the state-of-the-art of an entire field in a single human lifetime, let alone the implications of such things in a short span.
LLM's? No fucking way. We're well beyond ever explaining anything to anyone en masse ever again. From here on out it's going to be 'make up things, however you want them to sound, and you'll find you can get a majority of people believe you'.
I meant that the authors of the study should have gone much deeper, and WaPo should not have published such a lazy study.
I’d offer than much of the “AI” FUD in journalism is like this. Articles about dangerous cooking combinations, complaints about copyright infringement, articles about extreme bias.
This isn’t even AI FUD, it’s just bog-standard propaganda laundering by the Washington Post on behalf of the Intelligence Community (via some indirect incentive structures of Crowdstrike). This is consistent with decades of WaPo behavior. They've always been a mouthpiece of the IC, in exchange for breaking stories that occasionally matter.