# Enterprise RAG | Unlock Knowledge, Accelerate Decisions with Fireworks

> Fireworks enables enterprises to turn scattered documents, code, and data into actionable insights with fine-tuned retrieval-augmented generation. Deliver accurate answers, scale knowledge access, and reduce risk across your organization.

## Enterprise RAG for grounded answers

Fireworks is the model layer for enterprise RAG. Embeddings and reranking run on the same OpenAI-compatible API as your generation model, with Zero Data Retention by default. The vector store is your call.

[Talk to our team](https://fireworks.ai/contact) · [Get started](https://app.fireworks.ai/signup)

## Fireworks runs the models in your pipeline. Your vectors and documents stay in your store.

How enterprise RAG runs on Fireworks

[Talk to our team](https://fireworks.ai/contact) · [Get started](https://app.fireworks.ai/signup)

- **Inference**: Call Qwen3 Embedding 8B and serverless Qwen3 Reranker 8B on one OpenAI-compatible API, then move to dedicated deployments for Voyage. · [Embedding models](https://docs.fireworks.ai)
- **Training**: Training is optional for RAG. When a general embedding model misses your domain terms, bring a custom or fine-tuned one into your account and query it the same way. · [Bring your own model](https://docs.fireworks.ai)
- **Ensembling**: Ensembling barely touches retrieval. Nexus applies at the generation step, where you choose which model answers each request and hold a budget. · [About Nexus](https://docs.fireworks.ai/nexus)

## Open models for each RAG stage, under your data controls

Why use Fireworks for enterprise RAG?

- **One endpoint**: Every embedding model on Fireworks, Qwen3 or Voyage, answers at the same URL.
- **Hosted rerank**: Rerank with Qwen3 Reranker 8B before you commit to a dedicated deployment.
- **Open pricing**: Serverless embedding and rerank bill per million tokens at published rates.
- **Zero retention**: Fireworks doesn't log or store open-model prompts or outputs unless you opt in.

## What to look for in an enterprise RAG platform

Buyer's checklist

- **Permissions before retrieval**: Access filters run inside the vector query, so nobody retrieves a passage they can't open.
- **Hybrid search and reranking**: Keyword and vector recall feed a reranker that orders candidates before generation.
- **An eval set before scaling**: Real questions with expected answers, run before you widen the rollout.
- **Citations and refusals**: Grounded answers cite their passages and decline when retrieval comes back empty.
- **Data controls in writing**: Retention and encryption key controls, documented so your security team can review them.
- **Cost per answer visibility**: Per-token rates for each stage, so you can model what one answer costs before launch.

## How to set up enterprise RAG on Fireworks

How it works

[Read the docs](https://docs.fireworks.ai)

- **Step 1: Chunk and embed documents**: Split by document structure with metadata, then embed with Qwen3 Embedding 8B. Normalize the vectors before cosine scoring.
- **Step 2: Index in your vector store**: Write vectors and metadata to the store you already run. The docs cookbook uses MongoDB $vectorSearch.
- **Step 3: Filter by permissions first**: Apply each user's access filter in the vector query, then recall a broad candidate set.
- **Step 4: Rerank the candidates**: Send the query and candidates to /rerank with Qwen3 Reranker 8B and keep the top passages.
- **Step 5: Generate a grounded answer**: Pass the top passages to a generation model on the same API and instruct it to cite sources or decline.

## How does Fireworks handle your RAG data?

Security

Your queries and retrieved passages pass through Fireworks on every request. The data security docs set out what happens to them.

[Read data security docs](https://docs.fireworks.ai)

- **Logged, monitored data access**: Fireworks logs and monitors all customer data access, alongside the Zero Data Retention default for open models.
- **Encryption and your own keys**: Fireworks encrypts data in transit with TLS 1.2+ and at rest with AES-256. You can bring your own bucket and your own key.
- **Isolated, US-resident inference**: Dedicated workloads run in logically isolated environments, and you can keep serverless inference in the US only.
- **Certifications and compliance**: SOC 2 Type II, ISO 27001, 27701 and 42001, plus HIPAA compliance. Fireworks maps its controls to GDPR and CCPA.

## Which models can I use for enterprise RAG?

Retrieval models

Qwen3 Embedding 8B and Qwen3 Reranker 8B run serverless. The smaller Qwen3 sizes and the Voyage family run on dedicated deployments. Embedding models share one endpoint; rerankers use /rerank.

[See model docs](https://fireworks.ai/models)

- **[Qwen3 Embedding 8B](https://fireworks.ai/models/fireworks/qwen3-embedding-8b)**: Qwen · $0.1/M Tokens · 40,960 Context · LLM
- **[Qwen3 Embedding 4B](https://fireworks.ai/models/fireworks/qwen3-embedding-4b)**: Fireworks · 40,960 Context · LLM
- **[Qwen3 Embedding 0.6B](https://fireworks.ai/models/fireworks/qwen3-embedding-0p6b)**: Fireworks · 32,768 Context · LLM
- **[Qwen3 Reranker 8B](https://fireworks.ai/models/fireworks/qwen3-reranker-8b)**: Fireworks · $0.2/M Tokens · 40,960 Context · LLM
- **[Qwen3 Reranker 4B](https://fireworks.ai/models/fireworks/qwen3-reranker-4b)**: Fireworks · 40,960 Context · LLM
- **[Qwen3 Reranker 0.6B](https://fireworks.ai/models/fireworks/qwen3-reranker-0p6b)**: Fireworks · 40,960 Context · LLM

## How to keep answers grounded for enterprise RAG

Guide

### Apply permissions before retrieval

Store each passage's access list as metadata and filter on it inside the vector query, before ranking. A filter applied after generation comes too late, because the model has already read the passage. Re-index when permissions change, or a stale index can serve a passage to someone who just lost access.

### Retrieve wide, then rerank

Pull a broad candidate set from vector search, then let Qwen3 Reranker 8B order it against the query. One caveat for dedicated deployments: they scale to zero when idle, so the first request after a quiet period can return a 503. Build a retry for that call.

### Build an eval set first

Before you scale, ask the assistant's future users to write real questions, each with the answer they expect from a specific document. Rerun the set after every chunking or model change, and score refusals apart from wrong answers. Fix the wrong answers first.

## Learn more about enterprise RAG

Resources

[Browse the docs](https://docs.fireworks.ai)

- **[Understanding embeddings and reranking at scale](https://fireworks.ai/blog/embeddings-vs-reranking-rag-retrieval)**: 12 SEP, 2025 · Learn
- **[Building enterprise-scale RAG systems with Fireworks and MongoDB Atlas](https://fireworks.ai/blog/enterprise-rag-mongodb-atlas)**: 9 APR, 2025 · Announcement
- **[Building a RAG with Astro, FastAPI, SurrealDB and Llama 3.1](https://fireworks.ai/blog/rag-app-astro-fastapi-surrealdb)**: 14 AUG, 2024 · Learn

## Frequently asked questions about enterprise RAG

FAQ

[Talk to our team](https://fireworks.ai/contact)

### Does Fireworks support RAG pipelines?

Yes. Fireworks hosts embedding and reranking models for RAG and semantic search and serves your generation model on the same API. You pick the vector store.

### Should I fine-tune a model or use RAG?

Use RAG for facts that change or carry permissions, since you update the index instead of the model. Fine-tune for behavior and format. Fireworks supports both.

### Can I generate embeddings on Fireworks?

Yes. Send text to POST /inference/v1/embeddings in the OpenAI request format. Qwen3 Embedding 8B runs serverless, and Voyage models run on dedicated deployments.

### Is Fireworks a vector database?

No. Fireworks runs the models in your pipeline. You store and search vectors in the vector store you choose, such as MongoDB Atlas.

### What does enterprise RAG cost on Fireworks?

Serverless embedding and reranking bill per million tokens at the rates on the model cards. Your cost per answer adds up the tokens each stage processes.

### How does Fireworks handle my data?

Fireworks keeps no prompt or generation data for open models unless you opt in. The Security section above covers encryption and isolation.

## Ground every answer

Run embeddings, reranking and generation on one OpenAI-compatible API, with Zero Data Retention by default.

[Get started](https://app.fireworks.ai/signup/) · [Talk to an expert](https://fireworks.ai/contact)
