Great points and nice write-up.
At the risk of sounding AI pilled, I find LLM's are fairly good sparring partners if you instruct them well. At least they're more capable of giving you the best arguments against your position than the average bar patron.
Almost all of my non-professional prompts are of that nature.
I don't know. AI are basically trained to tell you what you want to hear, including when they're disagreeing with you. If you come off of your conversation with AI with a stronger belief than when you went in, is that merited?
"AI Advice Suppresses People’s Willingness To Say “I Don’t Know”, Even When The Advice Is Wrong And Accuracy Is Incentivized" https://osf.io/preprints/psyarxiv/5y6m4_v1
Yeah you have to frame the prompt as if you've come across some potentially extremist view you want to know more about, and pretend you're an outsider needing advice. Then you bias it to favor what you really want to know.