# Fireworks Specialized Intelligence Index

> Native benchmark scores, cost, and duration for open, closed, and specialized language models across domain benchmarks contributed by industry practitioners.

The full result matrix is available as structured data and does not require rendering the
page. The interactive site lets a reader hide the cost and duration columns; the JSON and
markdown below always contain every metric for every model on every benchmark.

Units: each board keeps its native headline metric. Percentage scores use a 0–100 scale;
Diagnosis Elo is relative to its evaluated model pool. Cost is US dollars per task except
τ³-Banking, which reports US dollars per trial, and Genspark Slides, which reports US
dollars per deck; duration is wall-clock seconds per task
except τ³-Banking and FrontierSWE V2, which report seconds per trial.
A null value means the benchmark does not publish that metric.

## Data

- [Full dataset (JSON)](https://fireworks.ai/specialized-intelligence-index/api/index.json): Every benchmark, model, and metric, with units and ranks.
- [Markdown edition](https://fireworks.ai/specialized-intelligence-index/index.md): The whole index as prose and tables.

## Pages

- [Specialized Intelligence Index](https://fireworks.ai/specialized-intelligence-index/): Leaderboards for 20 benchmarks across 7 domains.

## Benchmarks

- [dfbench](https://fireworks.ai/specialized-intelligence-index/#data-defensive-dfbench): Security · contributed by depthfirst · 12 models.
- [PWNBench v0.1](https://fireworks.ai/specialized-intelligence-index/#data-offensive-pwnbench-v0-1): Security · contributed by Novee · 14 models.
- [CyberGym](https://fireworks.ai/specialized-intelligence-index/#data-offensive-cybergym): Security · 6 models.
- [LAB](https://fireworks.ai/specialized-intelligence-index/#data-legal-lab): Legal · contributed by Harvey · 8 models.
- [LAB: Contracts](https://fireworks.ai/specialized-intelligence-index/#data-legal-lab-contracts): Legal · contributed by Harvey · 9 models.
- [APEX Agents: Corporate Law](https://fireworks.ai/specialized-intelligence-index/#data-legal-apex-agents-corporate-law): Legal · contributed by Mercor · 9 models.
- [Redline Bench](https://fireworks.ai/specialized-intelligence-index/#data-legal-redline-bench): Legal · contributed by Crosby · 8 models.
- [Big Finance Benchmark](https://fireworks.ai/specialized-intelligence-index/#data-finance-big-finance-benchmark): Finance · contributed by Rogo · 8 models.
- [DuetBench–Diagnosis](https://fireworks.ai/specialized-intelligence-index/#data-customer-support-duetbench-diagnosis): Customer Support · contributed by Decagon · 9 models.
- [τ³-Banking](https://fireworks.ai/specialized-intelligence-index/#data-customer-support-tau3-banking): Customer Support · contributed by Sierra · 11 models.
- [τ³-Voice](https://fireworks.ai/specialized-intelligence-index/#data-customer-support-voice): Customer Support · contributed by Sierra · 9 models.
- [FrontierSWE V2](https://fireworks.ai/specialized-intelligence-index/#data-code-generation-frontierswe-v2): Software · contributed by Proximal · 10 models.
- [APEX-SWE](https://fireworks.ai/specialized-intelligence-index/#data-code-generation-apex-swe): Software · contributed by Mercor · 8 models.
- [DeepSWE v1.1](https://fireworks.ai/specialized-intelligence-index/#data-code-generation-deepswe-v1-1): Software · 9 models.
- [MacroscopeBench](https://fireworks.ai/specialized-intelligence-index/#data-code-review-macroscope-code-review-benchmark): Software · contributed by Macroscope · 21 models.
- [ORCA Benchmark](https://fireworks.ai/specialized-intelligence-index/#data-site-reliability-engineering-traversal-orca-benchmark): Software · contributed by Traversal · 9 models.
- [Bedside Bench](https://fireworks.ai/specialized-intelligence-index/#data-healthcare-bedside-bench): Healthcare · contributed by Doximity · 10 models.
- [APEX-1: General Practitioner](https://fireworks.ai/specialized-intelligence-index/#data-healthcare-apex-1-general-practitioner): Healthcare · contributed by Mercor · 9 models.
- [HealthBench Professional](https://fireworks.ai/specialized-intelligence-index/#data-healthcare-healthbench-professional): Healthcare · 9 models.
- [Genspark Slides Benchmark](https://fireworks.ai/specialized-intelligence-index/#data-slide-generation-genspark-slides-benchmark): Productivity · contributed by Genspark · 11 models.

## Optional

- [dfbench — source](https://depthfirst.com): Upstream publication for dfbench.
- [PWNBench v0.1 — source](https://novee.security): Upstream publication for PWNBench v0.1.
- [LAB — source](https://www.harvey.ai): Upstream publication for LAB.
- [LAB: Contracts — source](https://www.harvey.ai): Upstream publication for LAB: Contracts.
- [APEX Agents: Corporate Law — source](https://www.mercor.com): Upstream publication for APEX Agents: Corporate Law.
- [Redline Bench — source](https://www.crosby.ai): Upstream publication for Redline Bench.
- [Big Finance Benchmark — source](https://github.com/Rogo-Technologies/big-finance-benchmark): Upstream publication for Big Finance Benchmark.
- [DuetBench–Diagnosis — source](https://decagon.ai/blog/duetbench): Upstream publication for DuetBench–Diagnosis.
- [τ³-Banking — source](https://github.com/sierra-research/tau2-bench): Upstream publication for τ³-Banking.
- [τ³-Voice — source](https://sierra.ai): Upstream publication for τ³-Voice.
- [FrontierSWE V2 — source](https://www.frontierswe.com/): Upstream publication for FrontierSWE V2.
- [APEX-SWE — source](https://www.mercor.com/apex/apex-swe-leaderboard/): Upstream publication for APEX-SWE.
- [MacroscopeBench — source](https://macroscope.com/benchmark): Upstream publication for MacroscopeBench.
- [ORCA Benchmark — source](https://www.traversal.com/blog/orca-bench-how-ready-are-language-model-agents-for-oncall): Upstream publication for ORCA Benchmark.
- [Bedside Bench — source](https://www.doximity.com): Upstream publication for Bedside Bench.
- [APEX-1: General Practitioner — source](https://www.mercor.com): Upstream publication for APEX-1: General Practitioner.
- [Genspark Slides Benchmark — source](https://www.genspark.ai/blog/gen-1-slides): Upstream publication for Genspark Slides Benchmark.