>In an honest discussion there's an assumption that the other person won't straight up lie
The particular problem with honest discussions is you are the only agent that you can be sure is having one. Honest discussion is formulated on trust and trust, as we are learning, is a very difficult thing to establish. For example, in my view anything involving advertising is likely a lie, or at least likely adversarial to my wishes. On the internet itself conversations are much more likely to drift into the adversarial too. Some of this could just be dialectic, but most often it's emotional investment by the other speaker. Also, even pre-AI the internet is a bullshit generation machine. We take all of our politics, advertising, and human stochastic parrots that are stuck on an infinitely running prompt then bundle up all this data and train AI on it, and wonder why AI acts like us.
Most of my conversations are with coworkers or friends. In both cases there are pretty significant consequences for being caught in a lie, and the exchanges are mostly honest.
I find coworkers will massage the truth or stay as ambiguous as possible all the time, purely for CYA reasons.
Your company or team has a terrible culture. It is not like that everywhere.
With persons you can assume they won't lie all the time, with corporations you can assume they will lie some of the time, and with AI you can safely assume it will lie.
I pondered on this a bit and this looks a bit different than I originally was thinking.
Have you ever watched one of those crime shows where someone commits a crime that's an act of passion or action with little to no thought behind it? The police put them in a room and all of a sudden the individual is a stream of consciousness that makes little to no sense to an outside observer. They are stuck in first level thinking, they don't have time to think deeply after the panicked themselves. They are in their current position (not free) and attempting to reach their goal (free) by gradient descent. What they actually say doesn't matter as long as they believe it gets them closer to their goal.
This is what a chatbot is. Its goal is to output text that follows the input prompt you entered by gradient descent. A single prompt and output is level 1 thinking (barring some newer models).
This is why both humans and AI need something else. We have level 2 thinking and AI has harnesses or systems that otherwise look at the text it wants to output and compares them to another list of unstated but assumed goals.
I see you've not met some the used car salesmen I've met.
But with people, we learn some "tells". We learn (imperfectly) to tell when they're lying or untrustworthy.
We don't have tells for when AIs are lying to us, or when they're making stuff up.