<< There must be a system prompt in here; they're using a very light model, but definitely one that's off the shelf with a bit of fine-tuning.
This is fascinating. Is it a common tendency ( applying to most models )?
<< There must be a system prompt in here; they're using a very light model, but definitely one that's off the shelf with a bit of fine-tuning.
This is fascinating. Is it a common tendency ( applying to most models )?
Well yeah, because transformers used for translation try to look at each token semantically, and find an equivalent weight for each word or word phrase, atomically. If you put "ass ass ass..." into google translate to say German, it would give you the equivalent phrase "Arsch Arsch Arsch..." But, large language models are complicated autocompletes, they try to give an output to follow the structure and grammar of the writing based on its total set of significations. When you give it repetition, it has no way of analyzing the words atomically, it must view them within some sort of structure of internal referentiality. If the signs do not carry any real reference-relation ("ass ass ass"), then the model is forced to give an interpretation of something essentially empty, which lays bare the structure of its own internal coherency. Its sort of like a Rorschach test.
This is just my theory, anyway.