Senior SWE-Bench

Engineering work taken from recent maintainer pull requests in projects such as PostHog, Gitea, and Electric: features asked for the way a product manager would, and bugs reported only by their symptoms. The agent sets up the repository itself and ships a patch. A solve counts as tasteful only when every verifier and validation test passes and the patch also clears judged bars for size, codebase practice, and code quality against the maintainer’s fix.

Last updated · Source leaderboard checked daily

Main Insights

Results

Senior SWE-Bench results, ranked by score (tasteful, pass@1), highest first. Last updated .
RankModelProviderScore (tasteful, pass@1)Cost (USD / task)Duration (seconds / task)
1Claude Opus 5.5Anthropic39.3%$3.071,543
2GPT-6 AstraOpenAI36.8%$2.321,029
3Claude Fable 5.1Anthropic34.7%$7.141,588
4Claude Opus 5Anthropic34.7%$9.282,772
5GPT-5.6 SolOpenAI34.7%$3.431,532
6Gemini 4 ArgonGoogle27.7%$4.962,970
7Grok 4.7xAI27.4%$3.851,694
8DeepSeek V4.1 FlashDeepSeek22.7%$0.747,005
9Kimi K3Moonshot AI20.2%$7.024,986
10GLM 5.2Z.ai17.9%$0.643,482