Join us for our inaugural conference, Forge 2026

FIREWORKS TRAINING

Beat the frontier
on your domain.

One platform to train, serve, and retrain your specialized models. Frontier lab-grade infrastructure, fully managed.

Start Training
Get Consultation
Kimi K3 SFT run

Choose your training path

Post-train models with built-in jobs or write your own. Fireworks Lab offers deep ML expertise, from co-design to custom build.

Placeholder

You run the loop

Run the loop from your own environment. Use the Training API to connect to Fireworks-orchestrated trainers and samplers. Start on Serverless. Request Dedicated when you need, provisioning required by Fireworks Lab.

  • For ML Researchers
  • SFT, DPO, ORPO, RL, distillation
  • LoRA (Serverless) to full-parameter (Dedicated)
Training API
Placeholder

We run the loop

Run built-in training jobs with Managed Training. Pick a method, pick a base model, launch from the UI or API.

  • For ML Engineers
  • Built-in training
  • SFT, DPO, RL
Managed Training
Placeholder

We build with you

Co-build your model with Fireworks Lab, frontier lab-level researchers and FDEs.

  • For all levels
  • Engagements range from co-design to custom build
  • Co-optimized training and inference
  • Custom evals
Fireworks Lab
Photo of person speaking to microphone
How it works

Watch a brief overview of your training path options with Fireworks

Build with Fireworks Lab

Co-build with the applied research and engineering team behind PyTorch, trusted by leaders at the frontier.

Consult with Fireworks Lab

Harvey

Post-trained Kimi K3 for long-horizon legal work using asynchronous RL. Harvey Tenet scored 19.7% all-pass on LAB against 11.5% for Claude Fable 5, while running at roughly a third of the cost per task.

Read more

Cursor

Built Composer 2 with mid-training and distributed async RL inside their own harness, and now tops CursorBench and SWE-bench Multilingual at 73.7 for six to ten times less than comparable frontier coding systems.

Read more

Heidi

Used SFT to teach an open model how their clinicians write, then RL to teach it what those clinicians approve, passing Gemini Flash and then Gemini Pro while cutting transcription latency from 25 seconds to seven.

Read more
YOU RUN THE LOOP

Training API

For ML researchers: Run the loop from your own environment. Use the Training API to connect to Fireworks-orchestrated trainers and samplers. Start on Serverless, graduate to Dedicated when you need.

Write the loop
  • •Decide the objective, build batches, set hyperparameter schedules, and wire up your data pipeline and experiment tracking.
  • •A cookbook recipe gives you a working loop for SFT, DPO, GRPO-style training, or experimental async RL, plus common loss functions and reward plumbing, and exposes learning rate, gradient accumulation, context length, and W&B as config.
Connect to compute
  • •Choose Serverless when speed to experiment matters most, or Dedicated when scale, control, and GPU economics take priority. Request Dedicated access from Fireworks Lab.
  • •Use a Cookbook recipe to handle resource setup and health checks for you.
  • •Fireworks provisions GPUs, manages the cluster, and runs the trainer lifecycle: create, health-check, reconnect, delete. (On serverless, you attach to an always-on pool.)
Train
  • •Call forward_backward for a built-in loss, or forward_backward_custom when the loss is yours.
  • •The GPU returns per-token log-probabilities with gradients attached, your function computes the scalar loss, and gradients go back for the backward pass. Then optim_step applies the update.
  • •Fireworks executes the distributed forward pass, backward pass, and optimizer step across nodes, including sharding and FSDP.
Sample
  • •Write the rollout, sampling, and evaluation logic your task needs, through the managed service.
  • •RL recipes ship deployment sampling and serving-integrated evaluation already built.
  • •Fireworks runs the inference deployment and, on dedicated, syncs weights onto it each checkpoint.
Checkpoint and promote
  • •Call save_weights_for_sampler and DCP snapshots, then promote the checkpoint you want to serve.
  • •A recipe handles save, promotion, and sampler refresh.
  • •Fireworks stores and exports checkpoints, and selects base versus delta snapshots automatically.
Tear down
  • •Serverless: No manual teardown needed; the API handles the training session lifecycle for you. Recipes clean up their own resources (setup, health checks, reconnect, cleanup).
  • •Dedicated: Close the service client as billing is time-based and runs until this is done. Recipes clean up their own resources, and Fireworks stops an idle trainer after 10 minutes of inactivity (configurable via inactivityTimeout, or disableable) to avoid runaway cost. Deployments aren't auto-cleaned; delete or scale them down.
  • •Both run types: Fireworks handles preemption recovery and job resume; you set the resume policy and make the state-restoration calls, while recipes carry resume logic for their own checkpoints.
Training API Compute Options
Serverless Compute (Self-Serve)Dedicated Compute (Talk to us)
ProvisioningAttach to an always-on shared poolTrainer and deployment provisioned for your run
BillingPer token, no idle GPU chargeTime-based on the GPUs you hold
ModelsA curated list of popular modelsUnlimited depth and breadth, up to the largest MoE models
Parameter modeLoRALoRA and full-parameter
ThroughputShared capacity and per-account rate limitsNo contention, no rate limits, scale up as much as you need

The platform built for reinforcement learning at scale

Fireworks is among the few organizations outside the frontier labs to have operated reinforcement learning (RL) across more than 10,000 GPUs. At scale, effective RL hinges on unified training and inference infrastructure that's purpose-built for correctness, efficiency, and development velocity.

Correctness

Train what you serve

For correctness in RL training, the rollout engine and trainer need to share the same numerical definition. With Fireworks, numerical formats match end to end across BF16, block FP8, and NVFP4. Kernel and reduction behavior are aligned to prevent numerical drift across training and inference. Router Replay preserves MoE expert selections between rollout and backward pass.

Efficiency

Maximize RL throughput

Fireworks is designed to keep GPUs doing useful work at speed across the RL loop. Asynchronous RL overlaps rollout generation with training to keep GPUs utilized. Hot-loading updated weights into running rollout deployments reduces time between training and rollout. For full parameter checkpoints, novel compression techniques are used to reduce the bandwidth and storage requirements for weight transmission, significantly accelerating the RL loop.

Velocity

Accelerate development

The bottleneck to production is often not the training job, but the time between iterations. Fireworks compresses development cycles from weeks to hours by turning train → deploy → evaluate → retrain into a continuous loop on one platform. Checkpoints deploy to production in one click, while production traces and evals feed the next run.

RL Rollouts

Keep your trainer. Run your rollouts at scale.

Fireworks handles the high-throughput, distributed inference across multiple clusters, so you can focus on improving model behavior instead of scaling low-level systems.

Explore RL Rollouts
WE RUN THE LOOP

Managed Training

For ML engineers: Pick a method, pick a base model, bring your data or evaluators. Run pre-built training jobs.

Select your training method

Three built-in methods. Which one fits depends on the data you already have.

  • •Supervised Fine-tuning — For tasks where you can provide verified examples of the desired behavior. Train text and vision models on input/output pairs.
  • •Preference Optimization (DPO / ORPO) — For goals you can judge but not measure, like style, helpfulness, or safety. Train on preferred response pairs.
  • •Reinforcement Learning — For goals you can measure with a reward, like reasoning accuracy, tool use, or task completion. Train against a custom reward function.
Choose your model

Select from our model catalog; eligibility is set per model and per method.

  • •The models matrix lists the methods each base model supports, its training shapes, and each shape's max context length.
  • •Managed training runs LoRA. Hundreds of adapters are servable from a single deployment.
  • •Warm-start from an earlier checkpoint instead of a base model when you're continuing a line of work.
Bring your data or evaluators

OpenAI-compatible chat completion format, so existing OpenAI SFT datasets run without conversion.

  • •SFT and DPO take labeled examples and preference pairs. RL takes prompts, with ground-truth completions optional, plus an evaluator that scores each rollout from 0 to 1.
  • •Evaluators can be rules, tests, an LLM judge, or a mix, and can call out to your own agent environment.
  • •Bring your own eval set, or let the job carve one out of your training data.
  • •Launch from the UI, from firectl and the API, or hand the job to your coding agent.
  • •We handle scheduling and checkpointing, and every checkpoint stays available as a starting point for the next run.
USE CASES

What would you like your model to do?

Find

Deep research · Enterprise search · Legal · Doc navigation

Train what to search, what evidence counts, and when to stop looking. "Finish with better evidence, not just more searches."

Explore research and search

Execute

Code repair · SQL · Infrastructure · Financial logic

The environment in the verifier. Run it, score it, train on what worked. "Let tests teach the model what working code looks like."

Explore code & executable

Act

Tool routing · MCP tools · API calls · Workflow agents

Train the decision layer between intent and action, including recovery. "Train the agent to finish the workflow, not merely call a tool."

Explore agents & tools

Judge

Legal review · Financial diligence · Clinical · Compliance

Train on the judgment calls your own experts make. Learn how a post-trained model catches 89%+ of risks in a deal-team report.

Explore domain judgement

Express

Writing style · Specialty formats · Classification · Extraction & schemas

Train on your vocabulary, your classifications, and the formats each specialty needs. "Your voice, your formats, every time."

Explore controlled output

Reason

Reasoning · Planning · Clarification · Multi-turn assistance · Critique

Train the model to plan, ask when the request is unclear, and check its own answer. "Reward the reasoning, not just the right answer."

Explore complex reasoning

Resources

Read these guides for the fundamentals of training on Fireworks. Follow the links inside each for per-model support, method detail, and infrastructure reference.

Getting started

Learn when training beats prompting, how SFT, DPO, and RL differ, and how to select the right surface. Also covers GPU quota by spending tier and data residency.

Training API

Learn about service-mode training, when to start from a recipe instead of the SDK, and how serverless compares to dedicated. Also covers datums, futures, checkpointing, and weight sync.

Managed Training

Learn how to launch a job from the UI, CLI, API, or your agent, and which models are eligible per method. Also covers custom model tunability and LoRA-only constraints.

Cookbooks

Find ready-to-run training recipes for GRPO, DPO, SFT, and distillation built on top of the Training API. Also covers install steps, W&B logging, and vision-language support.

Build your own frontier.

Train, serve, and continuously improve your own models.

Start training
Consult with Fireworks Lab