It is an interesting question. Of course everyone should have equal access to the data in theory, but I also believe nobody should be forced to offer it for free to others and I don't think I want to spend tax money having the government host and distribute that data.
I'm not sure how everyone can have access to the data without necessitating another taking on the burden of providing it.
I think torrent is a very good way to redistribute this type of data. You can even selectively sync and redistribute.
I'm also not saying anyone should be forced to disclose training data. I'm only staying that a LLM that's only openweight and not open data/pipeline barely fits the opensource model of the stack mentioned by OP.