Mixtral MoE 8x7B Instruct

Mixtral MoE 8x7B Instruct is an instruction-tuned sparse Mixture-of-Experts (MoE) model developed by Mistral AI. It fine-tunes the base Mixtral-8x7B model for conversational and instruction-following tasks.

The model is designed for:

• Conversational AI
• Code assistance
• Agentic systems
• Search and enterprise RAG
• Multimedia reasoning (text-only)

It outperforms Llama 2 70B across several benchmarks, per Mistral’s internal evaluations.

The model supports a context window of 32,768 tokens.

Fireworks supports the full 32.8K token context window on on-demand GPU deployments, with no rate limits.

• No function calling support
• No image input or multimodal support
• No embeddings or reranking capabilities
• Unmoderated outputs: No safety alignment or content filtering is applied

The model has 46.7 billion active parameters, drawn from 8 experts of 7B each, with 2 experts activated per forward pass.

• Standard fine-tuning: Not supported
• LoRA fine-tuning: Supported on Fireworks via Serverless LoRA

Tokens are counted across input + output, within the 32.8K context limit.

• Serverless: Not supported
• On-demand: Supported with no rate limits via dedicated GPUs

The model is licensed under the Apache 2.0 license, which allows unrestricted commercial use.