This is the real problem with LLMs. There is no way to separate code from data. At best, models could be trained on tokens that indicate untrusted data coming in. But then the untrusted tokens could also be messed with.
I've wondered if it would be possible for there to be two input streams: 1, for prompt, 2 for untrusted data. But I suspect that transformers would still only optionally decide what each one was for. So it would still be a prompt level suggestion, rather than a hard and fast rule.
My perception of real problem is that the LLMs were generic purpose tool and the focus was to improve their information retrieval and prediction. And they were fed with all this data (including private with was otherwise not available to everyone) for training purposes. The security and privacy of stored information was not really the requirement of this whole endeavor and all of sudden in the real world they are finding that this is a must requirement if they want to sell these models to enterprise companies.
And now all these security efforts to manage data privacy are akin to lipstick on a pig, they are brittle, costly, one-off. There are no boundaries inside the LLM storage, the training data is not encrypted at all in the memory across the pseudo tenants
You could have limited-instruction llms where the model does one thing, for example summaries. It could accept a limited amount of instructions for example, first token for verbosity, second for style etc...
LLMs should never be trained on restricted data of any kind, as we have seen that they are able to reconstruct their training data. The idea that they could be trained on private/restricted/copyrighted data and that was ok because there wouldn't be redistributing that data should have been killed 3 years ago.
Embedding vector indexes are how we separate code from data. Anything that is not for 100% unadulterated public access should be behind a traditional access control system. RAG search is not magic, it's just a SQL query of a manually created index. It absolutely could have access control built in. It's been out of laziness that it has not.
> The idea that they could be trained on private/restricted/copyrighted data and that was ok because there wouldn't be redistributing that data should have been killed 3 years ago.
I know, it's kind of mind-boggling. It's not old enough to be a history problem [0], and it's hardly a secret, so I have to assume it's a state of denial.
In other words, there are groups of developers and executives (and investor-money) with a collective motiviation to desperately shut their eyes and pray that Bad Thing simply won't happen to their product, at least not before someone else invents a revolutionary Silver Bullet that makes it go away.
[0] "Those who cannot remember the past are condemned to repeat it" -- George Santanya
I cannot disagree, but many who should know better do.
I have seen people argue with a straight face that there are no copyright concerns simply because of the sheer volume of the data that LLMs are trained on.
This makes less than zero sense. If someone has seen code, or heard music, and creates something too similar, it is a copyright violation, even though that person has seen much code or heard much music before. This is why the concept of "clean room" implementation exists, and why the concept of the abstraction-filtration-comparison legal text exists.
LLM proponents will point to the fact that courts have ruled that using copyrighted material for training has been ruled fair use.
This actually makes sense. Just as you can read a book, so can an LLM.
The thing that, AFAIK, hasn't been ruled on yet, is when the LLM regurgitates something that is too close to the book. If a human were to do that it is a clear copyright violation.
To pretend that "dilution is the solution to pollution" in terms of LLM training data, and that anything the LLM produces is original material, is to give LLMs more rights than humans have.