Crazy writeup.
Author is right about the base64 part. Does seem weird that it can decode and understand it at same time. And I guess what makes it weird that we just sorta accept that for say English and German this works ie normal use but when framed as base64 then it suddenly stops feeling intuitive
why tho? it's just an alternate alphabet/set of symbols.
Because its generally expected that models only work 'in distribution', i.e. they work on stuff they have previously seen.
They almost certainly have never seen regular conversations in Base64 in their training set, so its weird that it 'just works'.
Does that make sense?
For all we know, AI tech companies could theoretically have converted all of the "acquired" (ahem!) training set material into base64 and used it for training as well, just like you would encode say japanese romaji or hebrew written in the english alphabet.
Unlikely that every company would have bothered to do this.
'Yes, I know we already trained on all that data, but now I want you to convert to base64 and train it again! at enormous cost!'
If you do not properly MIME-decode email, you end up with at least some base64-encoded conversations.
[dead]