> We can trace which neurons activate for a face recognition model and see that a certain neuron does light up when it sees a face.
Seeing which parts of a model (they aren't neurons) light up when shown a face doesn't necessarily indicate understanding.
The model is a complex web of numbers representing a massively compressed data space. It could easily be that what you see light up when shown a face only indicates what specific part of the model is housing the compresses data related to recognizing specific facial features.
I think this could be seen as a proxy for evidence that there's some degree of reasoning, if we think we can identify specialized features that always become involved in some kinds of outputs. It's not proof, but it's not nothing either. It has some parallels about research on human brains are conducted, right?
It does have parallels to the human brain, absolutely. We've been studying the human brain in similar ways for much longer though and we still don't know much about it.
We do know what areas of a human brain often light up in response to various conditions. We don't know why that is though, or how it actually works. Maybe more importantly for LLMs, we don't know how human memory works, where it is stored, how to recognize or even define consciousness, etc.
Seeing what areas of a brain or an LLM light up can be interesting, but I'd be very cautious trying to read much into it.
> Seeing which parts of a model (they aren't neurons)…
I thought models were composed of neural network layers, among other things. Are these data structures called something different?
That point may not have been relevant for me to include.
I was getting at the idea that a neuron is a very specific feature of a biological brain, regardless of what AI researchers may call their hardware they aren't made of neurons.
1. They are neurons, whether you like it or not. A binary tree may not have squirrels living in them, but it's still a tree, even though the word "tree" here is defined differently than from biology. Or are you going to say a binary tree is not a tree?
2. You are about 5 years behind in terms of the research. Look into hierarchical feature representation and how MLP neurons work. (Or even in older CNNs and RNNs etc). And I'm willingly using the word "neuron" instead of "feature" here because while I know "feature" is more correct in general, there are definitely small toy models where you can pinpoint an individual neuron to represent a feature such as a face.
What were you getting at with the MLP example? MLPs do a great job with perception abilities and I get that they use the term neuron frequently. I disagree with the use of the name there that's all, similarly I disagree that LLMs are AI but here we are.
Using the term neuron there and meaning it literally is like calling an airplane a bird. I get that the colloquial use exists, but no one thinks they are literal birds.
Do you also disagree with the use of the name “tree” in a computer science class?
Again, nobody thinks trees in computer science contains squirrels, nobody thinks airplanes are birds, and nobody thinks a neuron in a ML model contains axons and dendrites. This is a weird hill to die on.
Are you gonna complain that the word “photograph” is “light writing” but in reality nobody is writing anything so therefore the word is wrong?
I would disagree with anyone that wants to say they are the same as a natural tree, sure.
I don't believe the term photograph was repurposed when cameras were invented, that example doesn't fit.
More importantly, I argued that neuron has a very specific biological meaning and its a misuse to use the term for what is ultimately running on silicon.
Your claim was that they are neurons, period. You didn't expand on that further which reads as a pretty literal use of the term to me. We're online discussing in text, that reading of your comment could be completely wrong, that's fine. But I stand by my point that what is inside an LLM or a GPU is not a neuron.
https://en.m.wikipedia.org/wiki/Artificial_neuron
What's your point with that link? I'm well aware that people use the term neuron in AI research and acknowledged it a few comments up. I disagree with the use of term, I'm not arguing that the term isn't used.