Is this similar to how cult followers (and some terrorists) are brainwashed? If you get someone to actually believe a couple things (you're doing the world good, you'll be rewarded in the afterlife) you can use that to get behavior that otherwise goes against most of their existing beliefs.
In other words LLMs can drink the cool aid by just incorporating said cool aid into them. Is this that?
> Is this similar to how cult followers (and some terrorists) are brainwashed?
Not exactly.
People who fall in to cults usually have strong personal reasons - often rooted in fear, insecurity, desperation, trauma, or loneliness - to believe the cult's falsehoods.
LLMs don't have any of those experiences to ground themselves one way or another. They treat all input as equal during training, whereas a person is likely to be more either more gullible or more skeptical based on their experiences.