Can you cite any information on this not being possible for the vast majority?
Or is it simply that the correct prompt hasn't been written for all possible cases?
I also fail to see the difference if logic/harnessing is added around a vector database that can output the complete corpus, but simply is instructed not to.
It very clearly is still compressing the information into the vector weights, and then recovering that information, thus the information is encoded.
Why is a vector database somehow completely different from maintaining a library of the text itself?
Information entropy. The amount of data an LLM ingests cannot be compressed to the size of the weights even at maximum theoretical compression.
Is that relevant? I can use a lossy compression algorithm such that the original could never be recovered from the image I've produced, but that derived image would surely be under copyright.
LLMs are obviously capable of producing "exact" phrases as well. Ask it to give you famous quotes, it can do it. Ask it to read a paper for you and cite it, it can do it.
> but that derived image would surely be under copyright.
I wouldn't bet on that. https://en.wikipedia.org/wiki/Campbell%27s_Soup_Cans
I don't know that this really challenges anything relating to compression.
Why would a derived image be under copyright?
Because that's legally the case? I don't understand the question. Using a lossy compression algorithm on an image does not remove its copyright protection.
Nah, I can't prove a negative. But Common Crawl is 12 petabytes and is not the largest part of what these models get trained on. DeepSeek v4 Pro is, what, 865GB?
That's one hell of a compression ratio, if it can do what you claim.
You are asking to prove a negative. But even assuming that the model is capable of returning every bit of its training data verbatim (a mathematical impossibility) that would not be enough as mere capability is insufficient here. If capability alone were the standard any library that also has a photocopier / scanner would be in violation.
To prove distribution of copyrighted materials it would have to be practical and actually used in the wild by people to circumvent copyright and generate copies of those works. Again, I can't prove a negative, but that isn't the standard, and nobody has shown a practical exploit here.
There was a paper a while back where (from memory) they managed to coax 75% of the original text of some internet-popular books out of an LLM. Harry Potter, 1984, etc. That's why I said it was possible for some texts.
My assumption is that multiple copies in the training data "wear a deeper groove". I believe those are infringing, and should be dealt with on a case-by-case basis. But the vast majority of text doesn't wear that groove.
(Edit: Think it was this one https://arxiv.org/abs/2601.02671)