sudo L7

Sixty staff-level engineering assignments written by senior engineers, mostly in private production codebases: requests like “make this faster” or “migrate these tokens safely” where the agent has to explore the system, find the hidden risks, and decide what should and should not ship. Each model runs in its developer’s own coding agent, and graders score the work against an expert rubric across eight engineering dimensions, from correctness to judgment and truthful reporting.

Last updated · Source leaderboard checked daily

Main Insights

Results

sudo L7 results, ranked by score (pass rate), highest first. Last updated .
RankModelProviderScore (pass rate)Cost (USD / task)
1Claude Fable 5.2Anthropic52.2%—
2Claude Sonnet 5.5Anthropic50.6%$6.54
3Claude Opus 5.5Anthropic46.1%$10.09
4GPT-6.1 SolOpenAI33.9%$0.91
5Claude Fable 5.1Anthropic32.2%$8.61
6GPT-6 AstraOpenAI31.1%$5.83
7MiniMax M3.1 FlashMiniMax18.9%—
8Kimi K3.1 ProMoonshot AI16.7%$3.04
9Qwen 3.8 MaxAlibaba16.7%$2.54
10Gemini 4 ArgonGoogle16.1%$2.64
11Muse Wonder 1.0Meta16.1%$4.39
12Hy4 PreviewTencent15.6%$0.72
13Inkling 2 SmallThinking Machines14.1%—
14Mistral Large 4Mistral13.9%$1.44
15Muse Spark 1.3Meta12.8%$1.85
16Grok 4.7xAI12.2%$4.43
17Kimi K3Moonshot AI12.2%$2.82
18Gemini 3.8 FlashGoogle10.6%$2.47
19GLM 5.3Z.ai9.4%$3.54