The saddest part is when people take their experience with Google's idiotic AI implementation and assume that's how all LLMs work. Frontier-class models will, in fact, generally admit when they don't know something. That includes the one I run at home on my own graphics cards, but it seems that Google just doesn't GAF.
Your point about "predicting the next word" mostly means that your post was very easy to predict.