It's worth noting that OpenAI recently came out saying, "We don't think SWEBench Pro is worth reporting any more" - https://openai.com/index/separating-signal-from-noise-coding...
It's worth noting that OpenAI recently came out saying, "We don't think SWEBench Pro is worth reporting any more" - https://openai.com/index/separating-signal-from-noise-coding...
lol
People will downvote you because this comment is "not appropriate" for HN, but there were countless conversations on HN about how important these benchmarks are.
I am literally LOLing at HN right now