ServerlessLLMChat
Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. They are text-to-text, decoder-only large language models, available in English, with open weights, pre-trained variants, and instruction-tuned variants. Gemma models are well-suited for a variety of text generation tasks, including question answering, summarization, and reasoning. Gemma 2 9B Instruct is the instruction-tuned version of Gemma 2 9B and has the chat completions API enabled.
Gemma 2 9B Instruct is available via Fireworks' serverless API, where you pay per token. There are several ways to call the Fireworks API, including Fireworks' Python client, the REST API, or OpenAI's Python client.
See below for easy generation of calls and a description of the raw REST API for making API requests. See the Querying text models docs for details.
Generate a model response using the chat endpoint of gemma2-9b-it. API reference
import requests import json url = "https://api.fireworks.ai/inference/v1/chat/completions" payload = { "model": "accounts/fireworks/models/gemma2-9b-it", "max_tokens": 1024, "top_p": 1, "top_k": 40, "presence_penalty": 0, "frequency_penalty": 0, "temperature": 0.6, "messages": [ { "role": "user", "content": "Hello, how are you?" } ] } headers = { "Accept": "application/json", "Content-Type": "application/json", "Authorization": "Bearer <API_KEY>" } requests.request("POST", url, headers=headers, data=json.dumps(payload))
On-demand deployments allow you to use Gemma 2 9B Instruct on dedicated GPUs with Fireworks' high-performance serving stack with high reliability and no rate limits.
See the On-demand deployments guide for details.