For anyone here who understands this stuff, help me out because the article dropped me at the end when drawing the connection from memory to LLMs.
For an LLM, the keys are the tokens in your context, and the values are … what exactly?
For anyone here who understands this stuff, help me out because the article dropped me at the end when drawing the connection from memory to LLMs.
For an LLM, the keys are the tokens in your context, and the values are … what exactly?
In an attention layer, the query, key, and value are all computed at inference time using weight matrices that are determined as a result of the training. If X is the token representations entering the attention layer, then each of Q, K, and V would be determined by multiplying X by the corresponding learned weight matrix for queries, keys, and values:
Q = XW_q
K = XW_k
V = XW_v
From there, you can think of K and V like keys and values in a Python dictionary, and Q as the key you pass (e.g. d[Q] in Python to return some value V). The difference in an attention layer is that the query is matched to each key using a dot product, and rather than getting a single value back you get a weighted sum of values.
> The difference in an attention layer is that the query is matched to each key using a dot product, and rather than getting a single value back you get a weighted sum of values.
I understand you meant to compare it to dicts, but this is more like (the only?) similarity to outer product memory. Worth calling out since GP is confused about this connection specifically.
Yes, the main problem with this whole way of thinking about transformers and self-attention is that when you look at how they are actually implemented (i.e. what Q, K, and V are applied to at each layer, and consider the loss function during training), then the analogy to memory quickly starts to look tenuous and far-removed from what is actually happening.
Mainly, since self-attention layers operate on deeply-stacked representations of the outputs of previous layers, and since training has a loss function that doesn't make the updates work in the way the article says, the analogy to memory and lookup tables breaks down quite strongly. IMO, thinking of Q, K and V as memory or key-value lookup is thus not actually very helpful for understanding what these layers are really doing or why they work. Others agree (e.g. https://news.ycombinator.com/item?id=46535412), so don't feel bad if you think there is something hand-wavey about the lookup-table and memory explanations - it is only a weak analogy.
This has come up on HN before, and probably it is most accurate (but less intuitive, sadly) to see self-attention as a generalization of kernel methods (https://news.ycombinator.com/item?id=46535549). Broadly, self-attention is just one of the simplest ways to model similarities / correlations between all inputs (i.e. kernel method), but there are many, many other attention mechanisms. There are also many other ways to implement memory (e.g. LSTMs), and many other architectures that might in fact have even more memory capacity than transformers.
That being said, this is still truly an excellent article and it really provides some great intuitions for why transformers might work as well as they do. The softmax / exponentiation here functioning to increase memorization capacity by suppressing weak activations is super clear and interesting, and I really appreciated it. I suspect is not really be what is happening or is actually important about it, given other research (https://arxiv.org/html/2410.18613v1), and because much of what works in DL is usually more about matrix conditioning and keeping gradients well-behaved generally, rather than what the actual bare final math / weights in isolation from training would tend to suggest. But it does give a great intuition for why a trained transformer can have the capacity it does.
Definitely! It's more of an intuition but doesn't explain what vector gets put in the weights and the analysis doesn't involve the learning dynamics at all. I link some papers at the end that do think about how memory relates to the dataset. But the architecture having this storing+lookup property probably is important in helping transformers remember so much information.
To clarify slightly, K and V are used to construct the dictionary, and then Q is the lookup key D[Q] as described (with the lookup being based on similarity rather than exact hash as it would be in an actual dictionary).
See my comments and links below, you are not wrong, IMO this is not actually a great analogy for understanding LLMs (other than as an explanation for why a trained one can so efficiently compress such a large amount of information).
https://news.ycombinator.com/item?id=50131741