# Conversational AI that Turns Knowledge into Action

> Unlock enterprise-scale Conversational AI with Fireworks. Fine-tune domain-specific models for research, reasoning, and branded content. Deliver fast, reliable, and context-aware insights that accelerate decisions, streamline collaboration, and power smarter workflows.

## Conversational AI that stays fast at turn 20

Fireworks is the inference platform for conversational AI. Prompt caching can cut time to first token (TTFT) by as much as 80% as chat history grows. The Responses API keeps state on the server.

[Talk to our team](https://fireworks.ai/contact) · [Get started](https://app.fireworks.ai/signup)

## How conversational AI runs on Fireworks

Prototype on serverless, then move to on-demand or Reserved Throughput when you need an SLA.

[Talk to our team](https://fireworks.ai/contact) · [Read the docs](https://docs.fireworks.ai)

- **Inference**: Iterate on serverless (no latency SLA), then run production chat on an on-demand Fast deployment with one minimum replica to skip cold starts. · [Deployment options](https://docs.fireworks.ai)
- **Training**: Once chat volume grows, fine-tune smaller models with LoRA; Cresta serves many adapters on one base model for up to 100x lower cost than GPT-4. · [Cresta story](https://docs.fireworks.ai)
- **Ensembling**: Send intent classification to a small model like Qwen2.5 7B and hard turns to a larger one. With Nexus, you decide which model answers each request and hold a budget. · [Qwen2.5 7B model](https://docs.fireworks.ai)

## Speed that holds across long chats and launch-day traffic

Why use Fireworks for conversational AI?

- **Notion speed**: Notion cut latency from about 2 seconds to 350 milliseconds, 4x faster.
- **Sentient launch**: 25–50% more concurrent users per GPU and sub-2s responses under peak load.
- **State on server**: The Responses API continues chats without resending the full history.
- **Model ownership**: Train open models on your chat data and own the model you ship.

## How to set up conversational AI on Fireworks

How it works

[Read the docs](https://docs.fireworks.ai)

- **Step 1: Get a key, try serverless**: Create an API key and send your first chat request to a serverless model to test prompts and answer quality.
- **Step 2: Carry the conversation**: Send prior turns in the messages array, or pass previous_response_id to the Responses API to keep state on the server.
- **Step 3: Stream every reply**: Set stream to true so tokens reach your UI as server-sent events while the model generates.
- **Step 4: Add tools**: Define tools in JSON Schema and set tool_choice to auto, none, required, or a specific function.
- **Step 5: Move to a Fast deployment**: Deploy on the Fast shape with --min-replica-count 1 to avoid cold starts, then read per-request TTFT from perf_metrics_in_response.

## Which models can I use for conversational AI?

Models

Use a small model to classify intent and larger open models for tool calls and reasoning turns. Check context and pricing on each live model card before you deploy.

[Browse models](https://fireworks.ai/models)

- **[Qwen2.5 7B Instruct](https://fireworks.ai/models/fireworks/qwen2p5-7b-instruct)**: Qwen · 32,768 Context · LLM
- **[Kimi K2 Instruct 0905](https://fireworks.ai/models/fireworks/kimi-k2-instruct-0905)**: Moonshot AI · 262,144 Context · LLM
- **[Deepseek V3 03-24](https://fireworks.ai/models/fireworks/deepseek-v3-0324)**: Deepseek · 163,840 Context · LLM

## How to keep long chats fast for conversational AI

Guide

### Cache the growing history

Prompt caching is on by default for every model and deployment, and growing chat history is one of its named use cases. Put the system prompt and tool definitions first and new turns last, so each request shares the longest prefix. Serverless cached tokens get a default 50% discount, which varies by model.

### Keep a session on one replica

The cache only helps if the next turn lands on the replica that holds it. Set the user field, the x-session-affinity header, or prompt_cache_key, and Fireworks sends that session's requests to the same replica. Without session affinity, turns on a busy deployment can miss the cache. Pin the session first.

### Measure TTFT on every request

Set perf_metrics_in_response to get TTFT, cached prompt tokens, and queue, prefill, and generation durations with each reply. Dedicated deployments ship with default speculative decoding drafters for most models. TTFT depends on your model, prompt length, and load, so measure it on your own traffic before you tune.

## Learn more about conversational AI

Resources

[Browse the docs](https://docs.fireworks.ai)

- **[Heidi x Fireworks: Bridging the gap in frontier model performance](https://fireworks.ai/customers/heidi-health-ambient-ai-scribe-fine-tuning)**: 20 JUL, 2026 · Heidi
- **[How Cresta drives millions of real-time, AI-powered contact center interactions with Fireworks](https://fireworks.ai/customers/cresta-knowledge-assist-multi-lora)**: 8 DEC, 2024 · Cresta
- **[20x faster Whisper than OpenAI - Fireworks audio transcribes 1 hour in 4 seconds](https://fireworks.ai/blog/fast-whisper-transcription-api)**: 9 DEC, 2024 · Announcement

## Frequently asked questions about conversational AI

FAQ

[Read the docs](https://docs.fireworks.ai)

### Does Fireworks keep conversation state?

Yes. The Responses API continues a chat from previous_response_id, so you skip resending the full history. With Chat Completions, you send prior turns in the messages array.

### Does Fireworks support streaming responses?

Yes. Set stream to true and tokens arrive as server-sent events. When you close the stream, generation stops and you aren't billed for tokens it never generated.

### Do I need function calling for a chatbot?

Only if your bot takes actions or reads live data. Fireworks supports OpenAI-compatible JSON Schema tools, tool_choice, and parallel tool calls on supported models.

### Does Fireworks support prompt caching?

Yes, by default on all models and deployments. It can cut TTFT on long chat histories, and the Guide above covers how to keep each turn on its cache.

### How much does chat on Fireworks cost?

Pricing is per model and per million tokens, listed on each model card. Serverless cached tokens are discounted, so long chats with cached history cost less per turn.

### Does Fireworks store my conversations?

Set store=false on Responses API calls to opt out of server-side storage. You then send prior turns yourself, as you would with Chat Completions.

## Stay fast at turn 20

Prompt caching keeps time to first token low as chat history grows, and the Responses API keeps state on the server.

[Get started](https://app.fireworks.ai/signup/) · [Talk to an expert](https://fireworks.ai/contact)
