This is one of the best writeups I've seen of this
a lot of the model's constraints come down to how they are RLed. Discussions online would be a lot better if everyone understood how the labs train the models in a high level (or did a lil data labeling)
yeah i like this and others in the thread mentioned that understanding RL and RLHF and the shape of the data is really important (at least the fundamentals, I'm sure there's quite complex industrialization of RL inside labs as Nathan Lambert says)