>You have to be rather naive if you don't think these companies don't simply train on your prompts with or without your consent. They literally scrape everything - legal or not - and claim its fair use to train on, including straight piracy

Have been having a think about this statement. I agree in intent, some of them are probably breaking the agreement for training data. I dont think Microsoft is doing it, Enterprise Data Protection is the plank holding up their entire Copilot line. One whiff and everyone's gone. Copilot isnt actually good at anything except giving some illusion of protection, and preventing users from following a desire path to other LLMs without enterprise data protection. Its the core value proposition. But we only need to wait and see what the next 20 data breaches tell us to find out for sure.

In detail however, I dont know if they could just claim it as fair use, after exclaiming that they specifically wont do that. I dont think "Fair Use" would be the issue so much as contract law. I know a EULA wouldnt hold up the other way (By reading this you agree not to steal my data and train an LLM with it) for data thats made freely available on the internet. But if you are purchasing the "No Training" contract they would be in breach if they trained with it. Possibly fair use would let them keep the data after paying whatever they owe in terms of contract breach.