You can also have a deep nuanced discussion with a rubber duck as long as you don't ask any questions it needs to respond to.

Have you not seen all the posts with claims that AI lies about its reasoning when asked to explain how it arrived at the output?

I would instead ask the model to explain how X works, whether it achieves Y, and why we cannot do Z instead.

That is how you have a discussion with the AI.

You can have a nuanced discussion with an LLM. But LLMs also have failure modes where they start making up justifications. The two are not mutually exclusive.

>as long as I don't ask how he arrived at his conclusions?

So just the average US political discussion with a human then?