Wouldn't it be possible to just fix a single activation, just set a continuous input to what the harness knows the role actually is. Models could then be trained to trust that input and not other signals about roles.
Wouldn't it be possible to just fix a single activation, just set a continuous input to what the harness knows the role actually is. Models could then be trained to trust that input and not other signals about roles.
I suspect there are many excellent solutions along these lines available to the labs training the models.
I wonder how well one could do on a conventional model with careful input formatting, e.g. JSONL where every line has bounded length and is something like:
It could need a bit of fine tuning to get this to work well.But I think the point is they learn "role" from the tone, and would ignore the JSON just like they ignore the tags already.
The models wouldn't need to adapt their training at all; this is a secondary model trained on their activations. It's electrodes stuck into the model's brain. The original article I posted is built on top of a regular LLM (OpenAI's GPT-OSS)