My favourite story of that involved attempting to use LLM to figure out whether it was true or my hallucination that the tidal waves were higher in Canary Islands than in Caribbean, and why; it spewed several paragraphs of plausibly sounding prose, and finished with “because Canary Islands are to the west of the equator”.
This phrase is now an inner joke used as a reply to someone quoting LLMs info as “facts”.
This is meaningless without knowing which model, size, version and if they had access to search tools. Results and reliability vary wildly.
In my case I can’t even remember last time Claude 3.7/4 has given me wrong info as it seems very intent on always doing a web search to verify.
It was Claude in November 2024, but the “west of equator” is a good enough universal nonsense to illustrate the fundamental issue - just that today it is in much subtler dimensions.
A not-so-subtle example from yesterday: Claude Code claiming to me yesterday assertion Foo was true, right after ingesting the logs with the “assertion Foo: false” in it.
There's something darkly funny about that - I remember when the web wasn't considered reliable either.
There's certainly echoes of that previous furore in this one.
> I remember when the web wasn't considered reliable either.
That changed?
There are certainly reliable resources available via the web but those definitely account for the minority of the content.
I think it got backgrounded. I'm talking about the first big push, early 90s. I remember lots of handwringing from humanities peeps that boiled down to "but just anyone can write a web page!"
I don't think it changed, I do think people stopped talking about it.
The web remains unreliable. It's very useful, so good web users have developed a variety of strategies to extract and verify reliable information from the unreliable substrate, much as good AI users can use modern LLMs to perform a variety of tasks. But I also see a lot of bad web users and bad AI users who can't reliably distinguish between "I saw well written text saying X" and "X is true".
> I remember when the web wasn't considered reliable either
It still isn't.
Yes, it still isn't, we all know that. But we all also know that it was MUCH more unreliable then. Everyone's just being dishonest to try to make a point on this.
I'm more talking about the conversation around it, rather than its absolute unreliability, so I think they're missing the point a bit.
It's the same as the "never use your real name on the internet" -> facebook transition. Things get normalized. "This too shall pass."