WitnessBench

Twenty-six unseen line-puzzle games inspired by The Witness, twenty levels each. The agent is never told a game’s rules: it has to discover them by experimenting, keep what it learns, and apply it as the levels get harder. A level counts only when the game engine accepts the agent’s line, and the score rewards solving it in close to the fewest actions the game allows.

Last updated · Source leaderboard checked daily

Main Insights

Results

WitnessBench results, ranked by score (oae-l20), highest first. Last updated .
RankModelProviderScore (OAE-L20)Cost (USD / task)
1Claude Opus 5.5Anthropic46.5%$85.39
2GPT-6 Astra ProOpenAI25.6%$58.88
3GPT-6.1 Sol ProOpenAI21.7%$11.81
4Kimi K3Moonshot AI18.8%$26.81
5Gemini 4 ArgonGoogle17.9%$17.72
6GLM 5.3Z.ai16.6%$32.62
7Grok 4.7xAI14.8%$21.08
8Gemini 3.8 FlashGoogle6.7%$7.41
9DeepSeek V4 Pro 0813DeepSeek5.5%$9.32
10Qwen 3.8 2.4T-A95BAlibaba5.0%$10.81
11Qwen 3.8 27BAlibaba3.1%$3.95
12Muse Glimmer 30BMeta0.2%$2.14
13Qwen 3.5 9BAlibaba0.1%$0.060