# Rollouts for RL

> Train, deploy, evaluate, and continuously improve custom models — all within one platform. Introducing the Fireworks Training Platform. This page explores reinforcement learning (RL) rollouts standalone service that enables large-scale RL rollouts at significantly faster speed and performance.

## Fireworks RL Rollouts

The only dedicated rollout inference for teams running their own RL. Get elastic capacity, production-grade inference, and sub-minute weight updates.

[Read the docs](https://docs.fireworks.ai/fine-tuning/rl-rollout-integration) · [Talk to our team](https://fireworks.ai/contact-training)

Trusted By The World's Leading Enterprises: Vercel, Lovable, Cursor, Cognition, Factory AI, Genspark, Uber, Doordash, HubSpot, Notion, Cresta, Heidi, Upwork, Quora, Samsung, GitLab, Micro 1, Trilogy, Juicebox, Lightfern, NinjaTech AI, UiPath, StackBlitz (Bolt)

## Cursor Improves Composer with Fireworks

Frontier Coding Intelligence

Cursor uses Fireworks to generate large volumes of realistic coding trajectories for reinforcement learning. Fireworks handles the high-throughput, distributed inference underneath the training loop, so Cursor’s researchers can focus on improving Composer’s data, rewards, and behavior.

[Learn more](https://fireworks.ai/blog/Cursor-Composer-2)

## Your trainer stays. The rollouts run on Fireworks.

Why run rollouts on Fireworks

Point your rollout fleet at Fireworks through OpenAI-compatible sampling and a weight update API, and we run the inference layer for RL at scale. You own training. We run the rollouts: production-grade serving, sub-minute weight updates, and numerics built for RL.

- **Faster, less expensive runs with optimized inference**: Run rollouts with the same industry-leading Fireworks inference stack for maximum throughput.
- **Delta-compressed updates**: Refresh rollout fleets with deltas, not full checkpoints. Typically ~2% of the model, swapped in under a minute.
- **Async RL, no idle GPUs**: Keep trainer and rollout workers running continuously instead of waiting on each other.
- **Multi-region capacity**: Scale across regions instead of standing up one large, co-located cluster.
- **Share capacity across product lines**: Use capacity across product lines like RL rollouts, production inference, or training to maximize utilization.

## Focus on RL, not rollout infrastructure.

| . | Build It Yourself | Fireworks |
| --- | --- | --- |
| Capacity | Manage massive, co-located clusters for training and rollouts | Included |
| Utilization | GPUs idle during synchronization | Included |
| Weight Updates | Build checkpoint distribution yourself | Included |
| Numerics | Maintain alignment yourself | Included |
| Team | Dedicated inference platform work | Included |

## Frontier teams run their rollouts on Fireworks

[View customer stories](https://fireworks.ai/customers)

> "I mean, the other alternative is we would build one in-house, but we have finite engineers like everybody else. We would prefer to have engineers make training more efficient and more precise rather than spin up an inference effort."
>
> — Federico Cassano, AI Researcher | Cursor

> "Vercel’s v0 model is a composite model. The SOTA in this space changes every day, so you don’t want to tie yourself to a single model. Using a fine-tuned reinforcement learning model with Fireworks, we perform substantially better than SOTA."
>
> — Malte Ubl, CTO | Vercel

> "Fireworks has been a key partner in helping us train and serve the models behind Cursor at scale. Their platform supports the high-throughput RL workloads and production inference required for Composer, giving us the speed, reliability, and efficiency to keep pushing the frontier of AI coding."
>
> — Sualeh Asif, CPO | Cursor

> "By partnering with Fireworks to fine-tune models, we reduced latency from about 2 seconds to 350 milliseconds, significantly improving performance and enabling us to launch AI features at scale. That improvement is a game changer for delivering reliable, enterprise-scale AI."
>
> — Sarah Sachs, AI Lead | Notion

## Or run the whole loop on Fireworks

Close the loop with Training

Not running your own trainer yet? Fireworks also runs fully managed RL and a Tinker-compatible SDK, so you can train and serve on one stack. Same kernels, same quantization, no handoff. What you train is what you serve.

[Explore training](https://fireworks.ai/product/training)

## Go deeper on the engineering

Fireworks Blog

[See more](https://fireworks.ai/blog)

- **[Frontier RL is cheaper than you think](https://fireworks.ai/blog/frontier-rl-delta-weight-updates)**: 22 MAR, 2026 · Learn
- **[Training-inference parity in MoE models: where numerics drift](https://fireworks.ai/research/moe-training-inference-parity)**: 10 MAR, 2024
- **[The fine-tuning bottleneck isn't the algorithm](https://fireworks.ai/blog/llm-fine-tuning-bottlenecks)**: 28 MAR, 2026 · Perspective

## Common Questions

FAQ

### Can I keep my own trainer?

Yes. Keep your trainer where it is, upload checkpoints to shared object storage, and use Fireworks for rollout serving and weight-update orchestration. Bring-your-own-trainer exposes OpenAI-compatible sampling, a weight update API, and status reporting.

### How fast are weight updates?

Delta-compressed updates are typically about 2% of the full model. Distribution across regions takes a few minutes, and the in-memory GPU swap stays under a minute because download and decode are pipelined ahead of the swap.

### How do you handle staleness?

Async RL tolerates a bounded, predictable off-policy delay. The systems layer keeps the gap small by making weight movement a routine background operation. Your algorithm tolerates the rest.

### What about MoE numerics?

The same kernels run across the trainer and rollouts. Divergences are published per model, and router replay aligns expert selection so training and sampling stay numerically consistent.

### How do I get started?

Rollouts run on reserved capacity. Start with a small proof of concept to validate the integration, then scale across regions. Talk to our team to size a deployment.

## Scale your RL rollouts on Fireworks

Point your trainer at production-grade rollout inference, or run the full loop on one stack.

[Read the docs](https://docs.fireworks.ai/fine-tuning/rl-rollout-integration) · [Talk to our team](https://fireworks.ai/contact-training)
