Mixtral MoE 8x7B Instruct

Mixtral MoE 8x7B Instruct is an instruction-tuned sparse Mixture-of-Experts (MoE) model developed by Mistral AI. It fine-tunes the base Mixtral-8x7B model for conversational and instruction-following tasks.

The model is designed for:

• •Conversational AI
• •Code assistance
• •Agentic systems
• •Search and enterprise RAG
• •Multimedia reasoning (text-only)

It outperforms Llama 2 70B across several benchmarks, per Mistral’s internal evaluations.

The model supports a context window of 32,768 tokens.

Fireworks supports the full 32.8K token context window on on-demand GPU deployments, with no rate limits.

• •No function calling support
• •No image input or multimodal support
• •No embeddings or reranking capabilities
• •Unmoderated outputs: No safety alignment or content filtering is applied

The model has 46.7 billion active parameters, drawn from 8 experts of 7B each, with 2 experts activated per forward pass.

• •Standard fine-tuning: Not supported
• •LoRA fine-tuning: Supported on Fireworks via Serverless LoRA

Tokens are counted across input + output, within the 32.8K context limit.

• •Serverless: Not supported
• •On-demand: Supported with no rate limits via dedicated GPUs

The model is licensed under the Apache 2.0 license, which allows unrestricted commercial use.