I hear the argument here, but isn't it possible it has dramatically more knowledge and when you get outside the common cases many of us use it for, it'll have completely different capabilities?
I feel like most benchmarks cluster on a reasonably limited area of human knowledge
Sort of depends on how well the core reasoning works. It’s not a big effort to connect an LLM to a search provider.
You do pay for the tokens, but in theory on a smaller model each token is cheaper.
honestly using search isn't that great, you mostly get SEO slop, it usually won't help the model ask the right questions
When I messed with it I used Kagi's search and I didn't have that issue (not claiming they're the best, they're the only one I tried).
They filter their results through their AI, though, so you get a sort of meta-summary of the top few results. It did well with geopolitical news stuff, but I've not tried a hard science sort of query.
Try Parallel.ai (no affiliation). Instead of keywords, the model writes objectives.