Enterprise RAG for grounded answers
Fireworks is the model layer for enterprise RAG. Embeddings and reranking run on the same OpenAI-compatible API as your generation model, with Zero Data Retention by default. The vector store is your call.
Fireworks is the model layer for enterprise RAG. Embeddings and reranking run on the same OpenAI-compatible API as your generation model, with Zero Data Retention by default. The vector store is your call.
How enterprise RAG runs on Fireworks
Call Qwen3 Embedding 8B and serverless Qwen3 Reranker 8B on one OpenAI-compatible API, then move to dedicated deployments for Voyage.
Training is optional for RAG. When a general embedding model misses your domain terms, bring a custom or fine-tuned one into your account and query it the same way.
Orchestration barely touches retrieval. Nexus applies at the generation step, where you choose which model answers each request and hold a budget.
Why use Fireworks for enterprise RAG?
Every embedding model on Fireworks, Qwen3 or Voyage, answers at the same URL.
Rerank with Qwen3 Reranker 8B before you commit to a dedicated deployment.
Serverless embedding and rerank bill per million tokens at published rates.
Fireworks doesn’t log or store open-model prompts or outputs unless you opt in.
Access filters run inside the vector query, so nobody retrieves a passage they can’t open.
Keyword and vector recall feed a reranker that orders candidates before generation.
Real questions with expected answers, run before you widen the rollout.
Grounded answers cite their passages and decline when retrieval comes back empty.
Retention and encryption key controls, documented so your security team can review them.
Per-token rates for each stage, so you can model what one answer costs before launch.
Split by document structure with metadata, then embed with Qwen3 Embedding 8B. Normalize the vectors before cosine scoring.
Write vectors and metadata to the store you already run. The docs cookbook uses MongoDB $vectorSearch.
Apply each user’s access filter in the vector query, then recall a broad candidate set.
Send the query and candidates to /rerank with Qwen3 Reranker 8B and keep the top passages.
Pass the top passages to a generation model on the same API and instruct it to cite sources or decline.
Your queries and retrieved passages pass through Fireworks on every request. The data security docs set out what happens to them.
Fireworks logs and monitors all customer data access, alongside the Zero Data Retention default for open models.
Fireworks encrypts data in transit with TLS 1.2+ and at rest with AES-256. You can bring your own bucket and your own key.
Dedicated workloads run in logically isolated environments, and you can keep serverless inference in the US only.
SOC 2 Type II, ISO 27001, 27701 and 42001, plus HIPAA compliance. Fireworks maps its controls to GDPR and CCPA.
Qwen3 Embedding 8B and Qwen3 Reranker 8B run serverless. The smaller Qwen3 sizes and the Voyage family run on dedicated deployments. Embedding models share one endpoint; rerankers use /rerank.
Store each passage’s access list as metadata and filter on it inside the vector query, before ranking. A filter applied after generation comes too late, because the model has already read the passage. Re-index when permissions change, or a stale index can serve a passage to someone who just lost access.
Pull a broad candidate set from vector search, then let Qwen3 Reranker 8B order it against the query. One caveat for dedicated deployments: they scale to zero when idle, so the first request after a quiet period can return a 503. Build a retry for that call.
Before you scale, ask the assistant’s future users to write real questions, each with the answer they expect from a specific document. Rerun the set after every chunking or model change, and score refusals apart from wrong answers. Fix the wrong answers first.
Yes. Fireworks hosts embedding and reranking models for RAG and semantic search and serves your generation model on the same API. You pick the vector store.
Use RAG for facts that change or carry permissions, since you update the index instead of the model. Fine-tune for behavior and format. Fireworks supports both.
Yes. Send text to POST /inference/v1/embeddings in the OpenAI request format. Qwen3 Embedding 8B runs serverless, and Voyage models run on dedicated deployments.
No. Fireworks runs the models in your pipeline. You store and search vectors in the vector store you choose, such as MongoDB Atlas.
Serverless embedding and reranking bill per million tokens at the rates on the model cards. Your cost per answer adds up the tokens each stage processes.
Fireworks keeps no prompt or generation data for open models unless you opt in. The Security section above covers encryption and isolation.
Run embeddings, reranking and generation on one OpenAI-compatible API, with Zero Data Retention by default.
