Fireworks is the open-model platform for search. Run open embedding and reranking models on one API, beside the model that writes the answer. Notion’s Head of AI Engineering: “Latency is perceived as search quality.”
Run every model hop on Fireworks and keep vectors in the store you already use.
Start on serverless Qwen3 Embedding 8B and Qwen3 Reranker 8B, then move to a dedicated deployment when you need the Voyage family.
When a generic model misses your latency budget, fine-tune a smaller open one, the way Notion cut latency from about 2 seconds to 350 ms.
Ensembling rarely helps search, since each hop calls a fixed model. If you ensemble models for answer generation and need to hold a budget, use Nexus.
What you get when one platform serves every model hop
Point your OpenAI client at Fireworks by changing the base URL and model name.
Embed at $0.10 and rerank at $0.20 per 1M tokens on serverless.
With RFT, Genspark beat a closed model by 12% on quality at half the cost.
Sentient got 25–50% more throughput per GPU for search under spiky load.
Create a Fireworks API key and set base_url to https://api.fireworks.ai/inference/v1.
Call fireworks/qwen3-embedding-8b on your documents. Set dimensions to fit your index and L2-normalize vectors for cosine.
Write vectors to the store you already run, then recall your top candidates with vector search.
Send the query and your top candidates to /rerank with fireworks/qwen3-reranker-8b. Results come back ordered by relevance.
Pass reranked results to a generator. Fine-tune it with LoRA when off-the-shelf answers miss your domain.
Start with serverless Qwen3 Embedding 8B and Qwen3 Reranker 8B, then add a generator. The Voyage family, including Voyage Multimodal 3.5, runs on dedicated deployments.
Time one query end to end: query understanding, embedding, vector recall, rerank and generation. Then repeat it at 20–50 parallel requests and read p95 and TTFT, since single-hop benchmarks hide the rest of the chain. Your user waits on the slowest hop.
On serverless, you pay per token for Qwen3 Embedding 8B and Qwen3 Reranker 8B. A dedicated deployment runs the Voyage family and scales to zero when idle, so the first request after idle can return a 503 during cold start. Build the retry in.
Live merge matches base-model latency, so pick it when one adapter carries your search traffic. Multi-LoRA loads many adapters onto one dedicated base deployment, which is how Cresta cut cost up to 100x vs. GPT-4. You pay with TTFT often 10–30% higher and lower throughput under load.
Yes. Qwen3 Embedding 8B runs serverless on the /embeddings endpoint. Qwen3 Embedding 4B and 0.6B and the Voyage family run on dedicated deployments.
On serverless, Qwen3 Embedding 8B is $0.10 per 1M tokens and Qwen3 Reranker 8B is $0.20 per 1M tokens.
Yes. Point the OpenAI client at https://api.fireworks.ai/inference/v1 and pass a Fireworks model name. Reranking uses a separate /rerank endpoint.
Start on serverless with Qwen3 Embedding 8B and Qwen3 Reranker 8B. Move to a dedicated deployment for the Voyage family, and retry the first request after idle, which can return a 503.
Fireworks doesn’t log or store prompt or generation data for open models unless you opt in. Data is encrypted in transit (TLS 1.2+) and at rest (AES-256).
Use search for retrieval quality and speed: embedding and reranking. Use Enterprise RAG for grounded generation over your internal documents.
Run open embedding and reranking models on one API, beside the model that writes the answer.
