People are gullible. LLMs generate tokens based on the previous tokens given to it. The LLM in Google's search box doesn't believe anything it was given; it is a Markov-esque chain that go from "Summarize the next sentences: $SEARCH_RESULTS" to the output.
I agree that there's a problem with searching today. The line between actual meaningful content and spam is blurring, all the meaningful indicators of the olden days to distinguish between good and bad contents are now gone/unreliable (polished proses, author's reputation). The signal/noise ratio is decreasing.
The approach to improving SNR should have been reducing/eliminating noise (flag spam sites, reputation system) and boost signal (also maybe reputation system, whitelist/blacklist). It's a hard problem simply because of entropy — the more content you have on the internet, the more random it will all seems from the top down.
I'm not saying I have the answer to this problem, I'm really just a noob when it comes to data science. I'm just thinking that mixing a bunch of text together and let a statistical model rehash that pile of grub into a professional, vindictive sounding response will *not* help providing users with enough signal to make sense of what they are looking for.