> There’s no reason to believe that kind of guess-and-check is outside of the reach of LLMs

This doesn't make any sense, by their nature they can't "guess-and-check" things outside their training set.