I think torrent is a very good way to redistribute this type of data. You can even selectively sync and redistribute.

I'm also not saying anyone should be forced to disclose training data. I'm only staying that a LLM that's only openweight and not open data/pipeline barely fits the opensource model of the stack mentioned by OP.