Fireworks RL Rollouts
The only dedicated rollout inference for teams running their own RL. Get elastic capacity, production-grade inference, and sub-minute weight updates.
The only dedicated rollout inference for teams running their own RL. Get elastic capacity, production-grade inference, and sub-minute weight updates.
Trusted By The World's Leading Enterprises
Cursor uses Fireworks to generate large volumes of realistic coding trajectories for reinforcement learning. Fireworks handles the high-throughput, distributed inference underneath the training loop, so Cursor’s researchers can focus on improving Composer’s data, rewards, and behavior.
Point your rollout fleet at Fireworks through OpenAI-compatible sampling and a weight update API, and we run the inference layer for RL at scale. You own training. We run the rollouts: production-grade serving, sub-minute weight updates, and numerics built for RL.
Run rollouts with the same industry-leading Fireworks inference stack for maximum throughput.
Refresh rollout fleets with deltas, not full checkpoints. Typically ~2% of the model, swapped in under a minute.
Keep trainer and rollout workers running continuously instead of waiting on each other.
Scale across regions instead of standing up one large, co-located cluster.
Use capacity across product lines like RL rollouts, production inference, or training to maximize utilization.
| Build It Yourself | Fireworks | |
|---|---|---|
Capacity | Manage massive, co-located clusters for training and rollouts | Included Elastic rollouts capacity across regions |
Utilization | GPUs idle during synchronization | Included Async RL keeps rollouts running |
Weight Updates | Build checkpoint distribution yourself | Included Delta-compressed updates built in |
Numerics | Maintain alignment yourself | Included RL-specific sampling alignment features to help ensure model updates are verifiable. |
Team | Dedicated inference platform work | Included Managed rollout infrastructure |
“I mean, the other alternative is we would build one in-house, but we have finite engineers like everybody else. We would prefer to have engineers make training more efficient and more precise rather than spin up an inference effort.”
Not running your own trainer yet? Fireworks also runs fully managed RL and a Tinker-compatible SDK, so you can train and serve on one stack. Same kernels, same quantization, no handoff. What you train is what you serve.
Yes. Keep your trainer where it is, upload checkpoints to shared object storage, and use Fireworks for rollout serving and weight-update orchestration. Bring-your-own-trainer exposes OpenAI-compatible sampling, a weight update API, and status reporting.
Delta-compressed updates are typically about 2% of the full model. Distribution across regions takes a few minutes, and the in-memory GPU swap stays under a minute because download and decode are pipelined ahead of the swap.
Async RL tolerates a bounded, predictable off-policy delay. The systems layer keeps the gap small by making weight movement a routine background operation. Your algorithm tolerates the rest.
The same kernels run across the trainer and rollouts. Divergences are published per model, and router replay aligns expert selection so training and sampling stay numerically consistent.
Rollouts run on reserved capacity. Start with a small proof of concept to validate the integration, then scale across regions. Talk to our team to size a deployment.
Point your trainer at production-grade rollout inference, or run the full loop on one stack.
