> So if the LLM is having a discussion with itself, am I paying for the tokens it uses for that?

You absolutely pay for them. This is why changing effort/reasoning levels have such a significant impact on session cost.