FrontierSWE V2

Thirty-four software-engineering tasks pitched at the edge of what an expert human can do: writing a flight-sim renderer in OpenGL, porting Git to Zig, driving a racing bot from vision alone. Each model gets five trials per task and up to twenty hours per trial, and every trial earns a graded reward rather than a pass or a fail, so a run that gets most of the way there still scores.

Last updated · Source leaderboard checked daily

Main Insights

Results

FrontierSWE V2 results, ranked by score (mean@5), highest first. Last updated .
RankModelProviderScore (mean@5)Cost (USD / trial)Duration (seconds / trial)
1Claude Fable 5.2Anthropic69.9%$168.7859,400
2GPT-6 AstraOpenAI65.5%$1029.6543,560
3Claude Opus 5.5Anthropic62.3%$98.8751,120
4Claude Sonnet 5.5Anthropic61.9%$110.4557,600
5Gemini 4 CarbonGoogle61.2%$61.9345,360
6Claude Fable 5.1Anthropic56.3%$138.5541,760
7Gemini 4 ArgonGoogle55.0%$129.3638,160
8Gemini 4 HydrogenGoogle53.1%$61.7941,040
9Claude Opus 5Anthropic52.0%$196.7157,960
10Claude Fable 5Anthropic47.0%$301.2856,880
11Claude Haiku 5.5Anthropic43.8%$20.5544,640
12GPT-5.6OpenAI32.2%$179.6430,960
13GLM-5.3Z.ai30.2%$97.2261,200
14Grok 4.7xAI29.5%$318.7743,560
15Kimi K3Moonshot AI25.9%$109.7166,240
16Muse Spark 1.3Meta25.8%$243.3461,200
17Grok 4.6xAI25.3%$243.4349,680
18Gemini 3.7 FlashGoogle20.3%$34.1428,440
19Gemini 3.8 FlashGoogle19.6%$38.7526,280
20GLM-5.3 FlashZ.ai18.1%$11.0359,040
21Qwen3.8-Max-0902Qwen17.8%$51.2769,840
22Qwen3.8-MaxQwen15.8%$55.1466,600
23DeepSeek V4 Flash Vision ExpDeepSeek14.8%$8.5753,280
24Muse Spark 1.2Meta12.0%$27.8116,560
25InklingThinking Machines4.1%$9.154,200