If they want to be open source, they also need to, y'know, open the source.
Weights are more like an executable binary. The source is the training data etc
If they want to be open source, they also need to, y'know, open the source.
Weights are more like an executable binary. The source is the training data etc
I don't understand this demand. Isn't the training data likely petabytes in size? You want that to be available? I imagine just the index (name,date,source) is a terabyte or two.
As the training data is every written thing they can get their hands on, right?
And wouldn't making that "open source" violate every license that the original data has? You can't release someone else's work.
Unless I have a misunderstanding?
I absolutely want that to be available.
The only way to make sure that a model was trained without hiding weird biases and backdoors is to monitor this process. It also allows anybody (with the hardware resources) to experiment with the model and how it's trained. Will most people want to do that? No, but most people also don't want to recompile their own kernel, that doesn't negate the value of Linux being FOSS.
I hope that, in the long term, I'll be able to use competitive, completely open models in the same way that I can run Linux on my computer.
You're completely right that the licensing of the training data is a big issue though (and I would argue that it should be a much bigger issue right now, even for closed models). I hope that eventually our understanding of the tech will let us build competitive LLMs using purely public domain/Creative Commons-style training data.
I realize that we have a long way to go... but that's a very worthwhile goal IMO.
Even if they did that, you would have no way of verifying that the data they upload and the model they claim to have trained is the weights they have shared. It’s a nonsensical idea.
If you want a trusted public model, we need to start a foundation that does it. The Linux of AI.
You’re going to need some serious money though.
I'm aware that in 2026 this is science fiction, but if the rate of computer improvement over the next 40 years matches that of the past 40, this is not going to remain sci-fi for extremely long in the grand scheme of things.
I hope that the long term future of AI is more like Linux and less like Windows.
You don't have access to the compute necessary to rebuild the original model even if all the original training data, custom human interventions, and RNG seeds were documented and released.
It is not a good faith request.
Also, what is this obsession with reproducing? Even if you could reproduce it, down to the RNG seed, that tells you nothing about whether or not some of the training data was malicious. You'd need an AI to go through the data and figure out if there is some malicious needle in a multi-petabyte dataset.
Releasing the model is good enough, because it is the only practical reality that makes sense.
To be clear I was making a wish for the future, I wasn't saying that Mistral were bad for not doing this today, that obviously wouldn't really be technically feasible.
Given the rate of progress of LLMs, I wouldn't be surprised if we had decent fully open and reproducible models within a decade or so, especially once the current spike in hardware price passes and building a local training cluster will be more realistic.
I don't usually compile my kernel from scratch, but I'm very glad that the source is open.
> And wouldn't making that "open source" violate every license that the original data has? You can't release someone else's work.
Tangent, and not the person you're replying to, but you just independently re-discovered why so many of us despise the AI companies.
Because that's what they are already doing. Selling a repackaged form of the stolen work of all of humanity, without credit or payment.
How else do you want to reproduce a model if you don't have the training data?
Just reproduce it? Copy the weights. What I think you want is the ability to improve it, to make what in copyright would be called a derivative work.
Reproduce meaning do the same process yourself and make sure you get the same output.
That isn’t possible, training is non-deterministic.
It doesn't have to be.
I don't recall scientific studies giving other scientists the original person's used in a study, set back to before the study. Ir the beakers they used.
You might find this comparison stupid, but I think it is rather apt.
Reproducing a work does not mean you get all the exact same things that were originally used.
At best, I would agree releasing the index is valuable for understanding how the model was made.
But frankly, I don't think the people who demand all this effort are arguing in good faith.
You do not have access to the compute necessary to "reproduce" what they made. So it is a pointless request.
Can you evolve binary like you can post train weights?
You can read and edit binary code just fine, it's much easier with AI too
Do you have examples you can point to?
For example there are more than 100k modified weights on hugging face.
Easy: https://www.nexusmods.com/
Modding isn't equivalent to post-training. Modders write new code, replace assets, patch existing instructions etc. With model weights you can continue original training process – directly optimizing existing artifact to acquire new behavior. There's no equivalent general-purpose process for a compiled binaries.
It's usually against the license to redistribute modified binaries. There are few pieces of software (I cannot think of any) that allow redistribution of modified binaries, but aren't open source.
As a result, in practice, the only example I can think of is cracks for pirated software.
Indeed.
You can hardly re-create the binary from a suitable source, just like the weights just fell from the sky. But you can change the binary (patch the machine code) just like you can adjust the weights. The comparison is quite apt.
The whole reverse engineering and cracking scene is built upon this premise, you know.
don't forget game modding. it's the only way it even works
Post training is general purpose method, same process that original authors used - that's not the case with binaries.
I think opening something in the hex editor, changing necessary bytes and saving it is as general purpose as it gets. You usually don't distribute it (I remember they days when it was common however) due to law, so in practice people write code that hot-patches stuff after starting the process.
You can also just open gdb and patch the process on the fly when you need to do things.
At the end of the day there isn't much hard difference between source code and binary code. By design source code is usually easier to read and write, but it's all syntax sugar and transformations all the way down.
Same as "airplanes are made to be used as submarines" – you just need hammer and few other tools and you're good to go.
RedHat and Ubuntu has kernel hot-patching. That's a case with binaries.
You're conflating runtime patching with continuing development.
Post training directly evolves existing model weights using the same optimization process that created them.
Software: source code -> compiler -> binary
Models: weights + training data + optimizer -> updated weights
Ie. binary is output of the development process.
Model weights are both – it's output and it's direct input for further development.
Model weight snapshots are continuously used all the time in this iterative process from the very beginning, being output of current step and input to the next step.
Calling it a "cracking scene" shows the different, though. Open weight models are intended to be used this way.
Hotpatching binaries used to be common. It's less so in the modern age but mainly because we moved software to higher levels of abstractions, virtual machines and interpreters, and also because rebuilding even a massive project from source is achievable on commodity hardware. You can still see various hotpatching schemes being developed, for instance to allow partial recompile of the sources while developping.
If in 2080 we can retrain Opus 129.5 directly on dev machines from scratch in <2h, I doubt many will bother messing with the weights directly.
Yes?
Hence the difference between open weights and open source.
But this is neither.