Fireworks is the open-model platform for search. Run open embedding and reranking models on one API, beside the model that writes the answer. Notion’s Head of AI Engineering: “Latency is perceived as search quality.”
Search is retrieval that matches a query’s intent with embeddings, often fused with keyword (BM25) results, then reranks the candidates by relevance.
You can build semantic product search, in-app search, knowledge retrieval for agents, deep-research search agents and recommendation engines. On Fireworks, the engineers adding semantic search to a product own the stack.
Llama 3.1 8B Instruct classifies the query’s intent before you embed and retrieve.
Qwen3 Embedding 8B turns documents and queries into vectors you store in your own vector database.
Qwen3 Reranker 8B reorders your top candidates through the /rerank endpoint and returns a relevance score for each.
Run every model hop on Fireworks and keep vectors in the store you already use.
Start on serverless Qwen3 Embedding 8B and Qwen3 Reranker 8B, then move to a dedicated deployment when you need the Voyage family.
When a generic model misses your latency budget, fine-tune a smaller open one, the way Notion cut latency from about 2 seconds to 350 ms.
Orchestration barely applies to search because each hop calls a fixed model; use Nexus if you orchestrate answer generation across models and hold a budget.
What you get when one platform serves every model hop
Point your OpenAI client at Fireworks by changing the base URL and model name.
Embed at $0.10 and rerank at $0.20 per 1M tokens on serverless.
With RFT, Genspark beat a closed model by 12% on quality at half the cost.
Sentient got 25–50% more throughput per GPU for search under spiky load.
Ask for latency on a full query, from query understanding to the generated answer.
Load-test at 20–50 parallel requests and track p95 and TTFT.
Generic embedders miss your vocabulary. Check that you can fine-tune the models in your path.
Vectors should load into your store. A Fireworks cookbook uses MongoDB $vectorSearch.
Add up tokens across every hop of a query. Per-token pricing lets you do that math before launch.
Ask what the first request after an idle period returns, and plan your retries around it.
Create a Fireworks API key and set base_url to https://api.fireworks.ai/inference/v1.
Call fireworks/qwen3-embedding-8b on your documents. Set dimensions to fit your index and L2-normalize vectors for cosine.
Write vectors to the store you already run, then recall your top candidates with vector search.
Send the query and your top candidates to /rerank with fireworks/qwen3-reranker-8b. Results come back ordered by relevance.
Pass reranked results to a generator. Fine-tune it with LoRA when off-the-shelf answers miss your domain.
Start with serverless Qwen3 Embedding 8B and Qwen3 Reranker 8B, then add a generator. The Voyage family, including Voyage Multimodal 3.5, runs on dedicated deployments.
Time one query end to end: query understanding, embedding, vector recall, rerank and generation. Then repeat it at 20–50 parallel requests and read p95 and TTFT, since single-hop benchmarks hide the rest of the chain. Your user waits on the slowest hop.
On serverless, you pay per token for Qwen3 Embedding 8B and Qwen3 Reranker 8B. A dedicated deployment runs the Voyage family and scales to zero when idle, so the first request after idle can return a 503 during cold start. Build the retry in.
Live merge matches base-model latency, so pick it when one adapter carries your search traffic. Multi-LoRA loads many adapters onto one dedicated base deployment, which is how Cresta cut cost up to 100x vs. GPT-4. You pay with TTFT often 10–30% higher and lower throughput under load.
Yes. Qwen3 Embedding 8B runs serverless on the /embeddings endpoint. Qwen3 Embedding 4B and 0.6B and the Voyage family run on dedicated deployments.
On serverless, Qwen3 Embedding 8B is $0.10 per 1M tokens and Qwen3 Reranker 8B is $0.20 per 1M tokens.
Yes. Point the OpenAI client at https://api.fireworks.ai/inference/v1 and pass a Fireworks model name. Reranking uses a separate /rerank endpoint.
Start on serverless with Qwen3 Embedding 8B and Qwen3 Reranker 8B. Move to a dedicated deployment for the Voyage family, and retry the first request after idle, which can return a 503.
Fireworks doesn’t log or store prompt or generation data for open models unless you opt in. Data is encrypted in transit (TLS 1.2+) and at rest (AES-256).
Use search for retrieval quality and speed: embedding and reranking. Use Enterprise RAG for grounded generation over your internal documents.
Run open embedding and reranking models on one API, beside the model that writes the answer.
