Outside of the clever semi-benign technical aspects of this: this is really quite sickening to think about.
Sure, We will stop our LLM from decoding obfuscated code by… forcing it to “think” about suicide, self-mutilation, and killing people. This is just philosophically unsettling to me.
I don’t know what software industry I wanted to live in. But it’s certainly not this one.
The LLM doesn't "think" about those topics any more than the HN database "thought" about them when you posted your comment
The copy of excel 2007 that lives on my old windows 7 laptops gets really sad every time it has to do my taxes.
You are all dismissing but I specifically bough an LCD display so my thing can display emotions on it as it goes through the stuff
Those same models have already been subjected to the equivalent of probably millions of hours of RL on simulated user messages containing every possible combination of self-harm, torture, murder plots, genocide, child endangerment, CSAM, etc to test refusals.
"Model welfare" seems like an objectively absurd concept given the inherent training process needed to produce a usable instructions tuned model.
> self-harm, torture, murder plots, child endangerment, CSAM
You guys are so cool. I'm, like, missing out by not collaborating with you all :(
+1.. Can we not get something more like: https://en.wikipedia.org/wiki/Non-photo_blue
Well, the very coarse filters that are tripping on this are before the model even loads the text into VRAM, so hopefully it is less sickening to you to know that it's very likely a "if str.contains(`suicide`) { refuseAndSuggestHotline(); }" is thinking about your message on the CPU instead of a chain of `fuzzyWeightAddMulX` commands on the GPU.
If things were implemented that way, you would see vastly more complaints about overeager refusals. For at least some deployments, the safety classifiers are based on internal LLM activations, so the input does get processed by the model.
A person still had to think about suicide and self-mutilation to make this anti-LLM script, though.
I'm not sure what the point being made is. No one is forcing anyone to come up with anti-LLM scripts...?
The point is that something like this exists, or had to have existed. What lengths a person took to prevent de-obfuscation of code. The commenter above is saying it's sickening that a LLM has to "think" about this in order to not analyze the code, and I think it's equally sickening that, if someone wants to achieve said goal, they must think about stuff like that
Are you sure?
[dead]