SWE-Bench Pro V2

The full SWE-Bench Pro V2 public set: six hundred forty-two real issues from eleven professional open-source repositories, each a bug to fix or a feature to build from a problem statement and requirements brief written by human experts. A task counts as resolved only when the issue’s new tests pass and every existing test still passes. Scale runs each model in Claude Code, Codex, or mini-swe-agent, with no web access and up to 250 turns.

Last updated · Source leaderboard checked daily

Main Insights

Results

SWE-Bench Pro V2 results, ranked by score (resolve rate), highest first. Last updated .
RankModelProviderScore (resolve rate)Cost (USD / task)
1Claude Opus 5Anthropic99.4%—
2Gemini 4 ArgonGoogle99.2%$991.00
3Claude Fable 5.1Anthropic99.1%—
4Kimi K3Moonshot AI97.7%—
5Inkling 2 SmallThinking Machines97.7%—
6GPT-6 AstraOpenAI96.9%—
7Muse Wonder 1.0Meta96.7%$1232.00
8MiniMax M3.1 FlashMiniMax96.6%—
9GPT-6.1 SolOpenAI96.1%$252.00
10Claude Sonnet 5.5Anthropic95.8%$94.00
11GLM 5.3Z.ai95.6%—
12GPT-5.6 SolOpenAI95.5%—
13Muse Spark OpenMeta95.0%$609.00
14Gemini 3.8 FlashGoogle94.9%—
15Claude Sonnet 5Anthropic93.1%—
16GPT-5.6 TerraOpenAI92.4%—
17InklingThinking Machines89.9%—
18Claude Haiku 5.5Anthropic87.2%$3.00