I’ve seen it directly contradict the citation so many times that i disregard the text and just click the citation or scroll past every single time. Just today i caught it making up the date for an event, and the citation had accurate information when clicked through.

It’s super easy to catch on dates and numbers, but it gets other details wrong all the time too. But so many people won’t be double checking the results.

> compared to the results in the actual Gemini.

Even those results have a lot to be desired, it is just buried deeper in the insanely verbose research report and impressive looking amount of sources you see move past.

I recently have had a close look at the various "deep research" options the big three (Anthropic, OpenAI and Google) offer. None of them are exactly transparent about how they perform searching other than the "research plan" the present upfront the and shitload of sources they show you (which, to be frank, seems to be clever UX/marketing to make it look extra legitimate and impressive). Which is already a worrying sign to me, as you can't audit the process itself properly. But even with the lack of information available on the front-end I can still see enough that worries me. A few examples:

- "Sources" are taken at face value almost no critical look at the validity of the source, the context it is placed in, etc.

- A lot of sources I know are legitimate are rarely included while a lot of listicles, low effort "reviews", etc do make the cut.

- In multiple instances when looking closer at the research plan and the "hints" they show during searching it becomes painfully clear that often enough they start with an answer in mind based on training data and try to validate that rather than actually researching the data itself.

- Subtly different prompts that by all means should still produce the same factual outcome actually provide wildly different results. This one probably relates to the other points.

In addition to all of this, I also am 100% convinced that AI powered search is incredibly expensive[1], more so than traditional search. In my mind this increased cost eventually will need to be paid by someone, which likely is going to be the user. Since the process is non-transparant I am not confident that the results will not end up being polluted by sponsored deals, etc. There is simply no way in my mind that this is going to end up well for us users.

[1] A while ago I have experimented with creating my own deep research flow with the idea that I might be able to do something with local models. To limit costs I used a SearXNG instance for searching, setup playwright for browsing sources. Using an agentic flow with agents making all the various calls and dispatching other agents ended up eating A LOT of tokens. Even when I did switch to a non agentic flow where each step is orchestrated by code calling on LLMs with simple prompts to validate results still ate a metric ton of tokens for the simplest search query. Mind you, this was not even doing actual deep research but only a few simple search queries. Ironically, google models also did seem to have more trouble coming up with good search queries compared to other models.

I want to add my interpretation of the phrasing "rewrite people's awareness" to make it read less tinfoil-hatted:

"rewire brains for AI dependency" (for money and power reasons obvious to everyone).

In contrast to "secretly implant an agenda about non-AI subject N", which is complicated enough that AI companies are still too out of control to be attempting yet.