At a glance
Challenge
- Making quality affordable at scale. Paper Instruments needed to bring the analytical, writing, and presentation capabilities of frontier closed models to an open model with inference costs suited to broader use in agents and applications.
- Professional judgment is difficult to train. Memos, analyses, and decks have no single correct answer. The team needed reliable training signals for what makes the work useful.
- Complex RL, limited engineering capacity. Scaling to GLM-5.3 Flash required timely model support and efficient compute without diverting the team from its data and research.
Solution
- Training for professional standards. Paper Instruments developed synthetic tasks and task-specific rubrics spanning four domains and 88 environment types.
- Research control through the Fireworks Training API. The team retained control over its data, sampling, loss functions, and training loop.
- Fast rollouts and efficient training. Fireworks combined optimized inference with GPU tuning for GLM-5.3 Flash, delivering rollout throughput that exceeded Paper Instruments’ expectations. The team projected Fireworks would offer the lowest cost per training step among the providers it considered.
- Training and inference engineered together. Fireworks addressed numerical discrepancies between the trainer and rollout engine that could distort learning, letting Paper Instruments focus on its data and RL methods.
Results
- Outscored Fable 5 at ~98% lower inference cost. Ultramarine scored 59.3 versus 57.8 at $0.13 versus $8.19 per task on Paper Instruments’ internal knowledge-work benchmark, giving the company a competitive offering on quality and price.
- A differentiated model for agent builders. Ultramarine gained 4.1 points on average over its base model across six knowledge-work benchmarks, giving Paper Instruments evidence to win over builders who might otherwise use the base model directly.
- Faster time to market. The team estimated that tuning GPU configurations itself would have delayed them by one to two weeks. Fireworks’ model-specific optimization helped Paper Instruments bring Ultramarine to market sooner.
Try Ultramarine 1 Flash on Fireworks. Available through serverless inference, it’s ready to test on your own analysis, writing, and presentation tasks.
Paper Instruments is building an open alternative to leading closed models for professional work. Its models are designed for AI agents and applications serving professionals in financial services, law, accounting, and business operations. The company aims to give the builders of those products professional-quality capabilities at substantially lower inference costs.
With Ultramarine 1 Flash, the team set out to teach an open model what good professional work looks like. Paper Instruments trained GLM-5.3 Flash with reinforcement learning (RL), developing its own data and rewards around analytical judgment, clear writing, and appropriate presentation.
The Fireworks Training API let the team scale its RL approach to GLM-5.3 Flash while retaining control over how the model learned. Fireworks delivered fast rollout generation, model-specific GPU optimization, and numerical alignment between training and inference, leaving Paper Instruments more time to develop its data and RL methods. Ultramarine outscored Fable 5 on Paper Instruments’ internal PI Knowledge Work benchmark at roughly 98% lower inference cost per task.
Turning professional judgment into a training signal
Paper Instruments needed a way to reward good professional judgment. For a board memo, there might be several defensible ways to explain a recommendation. A rubric built too closely around one reference answer could reject valid alternatives; a vague rubric could reward polished writing that missed a critical fact. The team had to make its grading precise without prescribing a single acceptable response.
Its researchers built synthetic tasks and grading criteria together, spanning four domains, 19 professional practice areas, and 88 environment types. They checked tasks repeatedly for realism, grounding, difficulty, and agreement between the assignment and its rubric.
In one published example, the model must update a board document after a forecast revision. It needs to carry the new figures through valuation calculations and distinguish what the board can approve from what still needs legal clearance. Completing the assignment requires reconciling the revised figures with the decisions and disclosures in the document.
Co-founder Daanish Khazi also wanted the model to recognize the presentation standards of its users. An investment bank might want simple, precisely aligned slides. Straightforward prose and restrained design were part of the training objectives alongside analytical quality.
Apart from a handful of spreadsheet tasks with directly verifiable answers, the training relied on rubric-based evaluation. The team tested different selections of data across hundreds of smaller-model experiments to develop the final curriculum. Khazi said most of the development budget went toward creating that data. Along the way, the researchers refined loss functions and length penalties as they developed their RL approach.
When GLM-5.3 Flash performed well in their evaluations, they had a promising base for Ultramarine. Moving the research onto that model brought a new set of engineering demands.
A five-person team takes on large-scale RL
Paper Instruments had been running its own training infrastructure for its smaller-model experiments. Moving to GLM-5.3 Flash required more specialized work on GPU configuration, parallelism, and model-specific kernels. Those choices affected training speed and how much experimentation the team could afford.
The team could do that work itself, but time spent tuning infrastructure was time it could have spent improving its training data and RL methods. The researchers also needed confidence in their experiments. If a run stopped improving, tracing the cause across data, training methods, and underlying systems could consume time and compute the team couldn’t spare.
Paper Instruments needed a training partner that could handle those systems demands while preserving its control over the research. Fireworks’ speed in making new models ready for training helped the team move from evaluating a promising model to putting its research into practice.
“We appreciate how quickly Fireworks validates and makes new models available for training. We know how much work goes into profiling, benchmarking, and checking numerical alignment, so that speed really matters to us.” - Daanish Khazi, Co-founder, Paper Instruments
The team then benchmarked providers that supported the model. Fast rollout inference mattered because each training cycle requires the model to generate responses for evaluation and learning. Khazi reported that Fireworks’ rollout throughput exceeded the team’s expectations. Based on those results, Paper Instruments projected Fireworks would offer the best cost per training step among the providers it evaluated.
More time for research, with infrastructure built for RL at scale
Paper Instruments used the Fireworks Training API and Fireworks inference to run the two sides of its RL loop: updating the model and generating the responses it learned from. Fireworks optimized both for GLM-5.3 Flash, bringing fast rollout generation, model-specific GPU tuning, and numerical alignment to the training process. The researchers retained control over sampling, loss functions, and asynchronous training, with dedicated compute allowing them to adjust rollout capacity as they refined their approach.
Khazi was particularly impressed by how Fireworks fit and ran the model on the available GPUs, describing the sizing and offloading work as “black magic.” He estimated that having the team tune those GPU configurations itself would have delayed it by one to two weeks, a meaningful setback in a market where model capabilities advance quickly.
“We retained full control over sampling and shaping the reward. We could focus on the RL science without having to think about kernel-level optimization, sizing, and parallelism.” - Khazi
That research freedom also depended on being able to trust the systems running each experiment. Fireworks develops and validates the trainer and rollout engine together, checking that they assign consistent probabilities to the same tokens when using identical model weights.
Those checks help catch numerical discrepancies that could distort training updates and look like problems with the data or learning method. For Paper Instruments, this meant more time refining its RL approach and less time investigating mismatches between training and inference.
Khazi also valued direct access to Fireworks engineers. When Paper Instruments needed lower serving latency, the Fireworks team identified a compatibility issue and made a tailored deployment configuration available that same day, with guidance on using live merge to serve its trained model.
Outscoring Fable 5 at ~98% lower inference cost
On Paper Instruments’ internal PI Knowledge Work benchmark, Ultramarine 1 Flash scored 59.3 versus Claude Fable 5’s 57.8, at $0.13 versus $8.19 per task.
The gains over the base model capture the impact of specialization. Ultramarine improved on GLM-5.3 Flash across all six reported knowledge-work benchmarks, gaining 4.1 points on average. The evaluations span professional deliverables, financial analysis, equity research, and business workflow automation. On Paper Instruments’ internal PI Knowledge Work benchmark, Ultramarine’s score rose from 52.0 to 59.3, a 7.3-point improvement.
Some of the most encouraging early feedback concerned how Ultramarine wrote. Khazi said financial-services users noticed more straightforward writing and simpler, more restrained presentation. Those responses reflected the professional standards the team had worked to teach, from clear prose in memos to precisely aligned slides without unnecessary visual flourishes.
After developing its curriculum through smaller-model experiments, the team completed the GLM-5.3 Flash training job on Fireworks in approximately two weeks. Total development cost was approximately $300,000, including the data creation and experimentation that preceded that run. The team concentrated its resources on what would differentiate Ultramarine: its data, professional standards, and RL methods, while drawing on Fireworks’ expertise to scale training and inference.
Start building with Ultramarine on Fireworks
Ultramarine 1 Flash is available in the Fireworks model library through serverless inference. With their Fireworks account and API key, developers can put it to work on analysis, writing, and presentations without provisioning GPUs or managing a deployment. Paper Instruments has also released the model’s weights, giving builders the flexibility to deploy it themselves.
For Khazi, working with partners like Fireworks makes it possible to build a company around model research. With Fireworks supporting training and inference, Paper Instruments can take its research from experimentation to customer use without building and operating the entire stack itself.
“Our business model would not work without an ecosystem of inference providers with optimized inference for different models.” - Khazi
The team is continuing to improve Ultramarine 1 Flash for enterprise work, building on the data, environments, and RL methods behind the model. It is also extending that approach to larger models and U.S.-developed base models.