NewJoin us for our inaugural conference, Forge 2026
DocsLog inGet started
Research

What we’re learning at Fireworks

Benchmarks, systems notes, and findings on training, inference, and model performance.

Kimi K3 on Fireworks: Fast RL at frontier scale
12 OCT, 2026

Kimi K3 on Fireworks: Fast RL at frontier scale

How Fireworks optimizes training, rollouts, and weight updates for reinforcement learning on Kimi K3, with serverless LoRA training for your own experiments.

Header Image Do Open Models Perform Silent Reasoning Before They Respond
12 AUG, 2026

Can open models carry readable silent signals before they speak? Reproducing J-Lens readouts on Kimi K3 & Qwen3.5-9B

FullFT vs LoRA training
30 JUL, 2026Training

Three tests to run before you switch from LoRA to FullFT

kernal-speedup-vs-minimax-MSA-oss
10 JUL, 2026

Optimizing MiniMax M3 sparse attention on NVIDIA Blackwell

  1. Kimi K3 on Fireworks: Fast RL at frontier scale

    12 OCT, 2026Research

  2. Can open models carry readable silent signals before they speak? Reproducing J-Lens readouts on Kimi K3 & Qwen3.5-9B

    12 AUG, 2026Research

  3. Three tests to run before you switch from LoRA to FullFT

    30 JUL, 2026Research

  4. Kimi K3 is competitive with Fable; Kimi K3 + Fable is SoTA.

    21 JUL, 2026Research

  5. Optimizing MiniMax M3 sparse attention on NVIDIA Blackwell

    10 JUL, 2026Research

  6. Frontier AI at a fraction of the cost: open-source workers with a closed-source advisor

    24 JUN, 2026Research

  7. The benchmark gap: what it takes to ship Kimi K2.5

    3 FEB, 2026Research

  8. Fireworks real-world benchmarks: find the best OSS model for the job

    30 JUL, 2025Research

  9. A deep dive into MLA training/inference difference and why QK-Clip from Kimi is such an elegant idea

    22 JUL, 2025Research

  10. VibeRL: When AI trains AI

    22 JUL, 2025Research

  11. Deep-dive into MuonClip: fixing attention score explosions in transformer training

    15 JUL, 2025Research

  12. DeepSeek V3 and R1 model architecture: why it's powerful and economical

    7 FEB, 2025Research

  13. Distillation with reasoning: can DeepSeek R1 teach better than humans?

    31 JAN, 2025Research

  14. Beyond supervised fine tuning: how reinforcement learning empowers AI with minimal labels

    27 JAN, 2025Research

  15. FireAttention V3: Enabling AMD as a viable alternative for GPU inference

    15 OCT, 2024Research

  16. How Fireworks evaluates quantization precisely and interpretably

    1 AUG, 2024Research

  17. FireAttention V2: 12x faster to make long contexts practical for online inference

    20 JUN, 2024Research

  18. Training-inference parity in MoE models: where numerics drift

    10 MAR, 2024Research

  19. FireAttention — serving open source models 4x faster than vLLM by quantizing with ~no tradeoffs

    8 JAN, 2024Research

  20. Speed, Python: pick two. How CUDA graphs enable fast Python code for deep learning

    29 AUG, 2023Research

  21. Multi-query attention is all you need

    12 JUL, 2023Research

Build your own frontier.

Start running open models in minutes. Then orchestrate, train and climb toward intelligence no one else can rent.

Get started
Talk to experts