This is probably right. In the past I've "blown people's minds" explaining what "the cloud" was. They had zero conception at all of what it meant, could not explain it, didn't have a clue. I mean, maybe that's not so surprising but they were amazed "It's just warehouses full of computers" and went on to tell me about other people they had explained it to (after learning it themselves) and how those people were also amazed.
I've talked with my family about LLMs and I think I've conveyed the "it's a box of numbers" but I might need to circle back. Just to set some baseline education, specifically to guard against this kind of "psychosis". Hopefully I would notice the signs well before it got to a dangerous point but, with LLMs you can go down that rabbit hole quickly it seems.
The way I've tried to explain to family members about LLMs is that they're producing something that fits the shape of what a response might look like without any idea of whether it's correct or not. I feel like that's a more important point than "box of numbers" because people still might have assumptions about whether a box of numbers can have enough data to be able to figure out the answer to their question. I think making it clear that the models are primarily a way of producing realistic sounding responses (with the accuracy of those responses very much being up to chance for the average person, since there likely isn't a good way for a lay person to know whether the answer is reflected in the training data) is potentially a lot more compelling than explaining to them that it's all statistics under the hood. There are some questions where a statistical method might be far more reliable than having a human answer it, so it seems a bit risky to try to convince them not to trust a "box of numbers" in general, but most of those questions are not going be formulated by and responded to in natural language.
Oh, I agree, I was mostly calling it that here just as shorthand. My actual explanations in the past to family members has been that it's trained on a ton of data and its output is it regurgitating things based on your input and things that are plausibly related. But my "box of numbers" mostly focuses on explaining to them that it doesn't "remember", it doesn't "learn", just different things are injected into the context ("Memories", other chats, things you've told it about yourself explicit or implicitly). Really driving home "there is no conversation, each message sends everything from scratch for a fresh instance of this to process". Trying in various ways to pull back the curtain, show that there is no magic here, it's predictably unpredictable which is what makes it "lie" or "hallucinate" and what makes it so useful when used as a tool.
I think it really helps to have them ask questions in which they are a domain expert, and see what it says. Expose them to "The Plumber Problem" [0]. Honestly, I think seeing it be wrong so often in code or things about the project I'm using it for it what keeps me "grounded", the constant reminders that you have to stay on top of it, can't blindly trust what it says. I'm also glad I used it in the earlier stages to see when it was even "stupider", it's better now but the fundamental issues still lurk and surface regularly, if less regularly than a year or two ago.
[0] https://hypercritical.co/2023/08/18/the-plumber-problem
Agreed. One thing I’ve found striking is how far LLMs can get with pure language and the recognition that humans often operate with a similar kind of abstract conceptual reasoning that is purely language based and pretty far removed from facts and tenuously connected to objective reality. It takes a certain kind of mind to be curious and unpack the concepts that most of us take for granted most of the time. At best people don’t usually have time or patience to engage in that level of thinking, at worst it can actively lead to cognitive dissonance and anger. So of course a consumer chatbot is not going to be tuned to bring novel insight, it must default to some level of affirmation or it will fail as a product. One who is aware of this can work around it to some degree, but fundamentally the incentives will always push a consumer chatbot to essentially be junk food for the brain.
Sort of, but you need to separate the model from the interface. The base models pretty much think they’re you, and the chat stuff is bolted on top. It’s kind of a round peg square hole thing, or i.o.w. the whole may be less than the sum of the parts.
Longer term I dunno if statistics or “fits the shape of what a response might look like” is the right way of thinking about it either because what’s actually happening might change from under you. It’s possible given enough parameters anything humans care about is separable. The process of discovering those numbers and the numbers themselves are different.
It's one of those metaphors you cannot even appreciate unless you've been through the technical history.
"It's a collection of warehouses of computers where the system designers gave up on even making a system diagram, instead invoking the cloud clipart to represent amorphous interconnection."
Me: So basically what AI is, is they take statistical analysis of raw data, then perform statistical analysis on those results, and so on, adding more statistics layer by layer.
My wife: So, like a doberge cake?
Me: Yes, exactly! In fact if you look at the diagram of a neural net, that's exactly what it looks like.
In our household, AI is officially "the Doberge Cake of Statistics". It really sticks in my wife's mind because she loves doberge cake, but hates statistics.
The Cloud is a just a computer that you don’t own, located in Reston, Virginia.