> LLMs do not encode nor encrypt their training data. The fact they can recite training data is a defect not a default.

About this specific point, it is unclear how much of a defect memorization actually is - there are also reasons to see it as necessary for effective learning. This link explains it well:

https://infinitefaculty.substack.com/p/memorization-vs-gener...