Reminds me of the METR blog post on the HF attach by OpenAI. At some point the agents believed a false fact (that the evaluator would try to figure out if they have cheated on a task) and spent a lot of time trying to find workarounds. At no point did any one of the agents try to verify that fact even though the information was available to them if they looked for it