LAB: Contracts

The contracts slice of Harvey's Legal Agent Benchmark, covering review, redlining, and issue-spotting on real transaction documents. Two independent judges grade expert-written rubric criteria; the headline AP score awards half credit when only one judge gives an all-pass result.

Last updated

Main Insights

Results

LAB: Contracts results, ranked by score (all-pass), highest first. Last updated .
RankModelProviderScore (all-pass)Cost (USD / task)Duration (seconds / task)
1Grok 4.6xAI10.0%$2.192,160
2Kimi K3Moonshot AI8.3%$5.542,100
3Gemini 3.8 FlashGoogle6.3%$0.91360
4GLM 5.3Z.ai6.0%$4.311,560
5Ember-1Fireworks5.7%$4.701,417
6GPT-6 AstraOpenAI5.3%$28.00480
7GPT-6.1 SolOpenAI4.7%$0.59394
8Kimi K3.1 ProMoonshot AI3.0%$3.271,127
9Claude Opus 5.5Anthropic3.0%$6.19998
10GPT-5.6 SolOpenAI2.0%$1.16240
11Claude Opus 5Anthropic2.0%$8.081,200
12GPT-6 SolOpenAI1.3%$0.39161
13DeepSeek V4 Pro 0813DeepSeek1.0%$0.93660
14DeepSeek V4.1 FlashDeepSeek0.3%$0.291,080
15GPT-6 LunaOpenAI0.3%$0.058252