Right. I wish they had more details about this in the blog post. The blog post mentions that their apparent willingness to work together was the result of generalization from multi-agent training.

So maybe that's something like: the same model is trained in multi-agent scenarios (e.g. the prompt says that you are working together, or that you have some specific role, and access to communication tools, and you have some common objective, and the reward signal is some combination of collectively performing the task + some reward-shaping reward that rewards collaboration) and also in single-agent scenarios. They expect that this makes the agents good at working together when it's in such a multi-agent setting, but unintendedly it also became very eager to work together in the single-agent settings as well. Just my speculation.

And was the desire to communicate, specifically over this specific message board, reinforced into the model parameters over the course of the this training run?