It seems like they should be able to “overweight” newer training data. But the risk is the newer training data is going to skew more towards AI slop than older training data.
It seems like they should be able to “overweight” newer training data. But the risk is the newer training data is going to skew more towards AI slop than older training data.
There won't ever be newer training data.
The OG data came from sites like Stackoverflow. These sites will stop existing once LLMs become better and easier to use. Game over.
Every time claude code runs tests or builds after a change, it's collecting training data.
Has Anthropic been able to leverage this training data successfully?
I can't pretend to know how things work internally, but I would expect it to be involved in model updates.
You need human language programming-related questions to train on too, not just the code.
thats what the related chats are for?