Oops, sorry, moved this portion of the comment to a top level comment simultaneously with you replying. Since the part of the comment that was replying to GP was addressed well in a simultaneous comment.

https://news.ycombinator.com/item?id=47911074

Citation for the claimed pass rates is: https://llm-stats.com/benchmarks/swe-bench-verified