This shouldn’t be surprising. Let’s start off with the obvious. What does “real-world fact-check claims” mean? So we’re using the same list of “fact check claims” on each model. The problem is (unless I’m missing it) the authors aren’t exposing the list of 1K questions they used in the experiment. That’s a huge problem. Are the authors assuming the 1K claims they used are “provably true”? If so, that’s a huge bias, and opens up a philosophical debate about what it a fact? Or what’s makes something true/ false?

As Marc Andreessen puts it: a particular domain is either explicitly “provable” or not “provable”. Provable domains include math, physics, chemistry, biology, engineering, even code. That not be the whole list, but everything else is essentially “unprovable”. At least as far as a language model is concerned. They are questions that require a human value judgement. Politics are an obvious example. So back to the “1K fact check claims“. How many of these are political, or current events questions? How many are STEM questions that can be laid out in a formal proof?

Models can be trained to answer either way on claims that require a value judgement, but that’s obviously not beneficial to anyone except who controls the model. If the expectation is that all these frontier models should answer the same way on value judgement questions, then that’s never going to happen. What the models ARE good at though is breaking down the nuances of a topic and arguing both sides. This is how these tools should be used, as a way to analyze the claim and let us humans in the end make our own value judgement. If you’re trusting the model to make the value judgement for you and just accept it as a fact, then you are entering a a very dangerous territory.