Oh, I agree, I was mostly calling it that here just as shorthand. My actual explanations in the past to family members has been that it's trained on a ton of data and its output is it regurgitating things based on your input and things that are plausibly related. But my "box of numbers" mostly focuses on explaining to them that it doesn't "remember", it doesn't "learn", just different things are injected into the context ("Memories", other chats, things you've told it about yourself explicit or implicitly). Really driving home "there is no conversation, each message sends everything from scratch for a fresh instance of this to process". Trying in various ways to pull back the curtain, show that there is no magic here, it's predictably unpredictable which is what makes it "lie" or "hallucinate" and what makes it so useful when used as a tool.
I think it really helps to have them ask questions in which they are a domain expert, and see what it says. Expose them to "The Plumber Problem" [0]. Honestly, I think seeing it be wrong so often in code or things about the project I'm using it for it what keeps me "grounded", the constant reminders that you have to stay on top of it, can't blindly trust what it says. I'm also glad I used it in the earlier stages to see when it was even "stupider", it's better now but the fundamental issues still lurk and surface regularly, if less regularly than a year or two ago.