that's good, open weight models do not need degenerate comments to advance on AI
It would be interesting to hear if the addition of Reddit data makes LLM stronger or weaker.
Outside of some (technical)subs most of the comment seems to be of a far lower quality than what you find in books or in websites.
For years before LLMs got big a comment I would often see was, "Google search is worthless now. I basically solve all my problems by searching reddit."
Someone did a finetune w/ data from 4chan and found improvements on several popular benchmarks, so ... maybe?
It would be interesting to hear if the addition of Reddit data makes LLM stronger or weaker.
Outside of some (technical)subs most of the comment seems to be of a far lower quality than what you find in books or in websites.
For years before LLMs got big a comment I would often see was, "Google search is worthless now. I basically solve all my problems by searching reddit."
Someone did a finetune w/ data from 4chan and found improvements on several popular benchmarks, so ... maybe?