​

Reward Kit API Reference

This API reference provides detailed documentation for the key classes, functions, and data models in the Reward Kit.

​

Core Components

​

Classes and Decorators

• RewardFunction Class: Core class for wrapping and calling reward functions
• reward_function Decorator: Decorator for creating deployable reward functions

​

Data Models

• Data Models: Documentation for Message, EvaluateResult, MetricResult, and other data models

​

Modules

​

reward_function Module

The reward_function module contains the core functionality for creating and using reward functions.

from reward_kit.reward_function import RewardFunction, reward_function

​

evaluation Module

The evaluation module provides the Evaluator class for managing evaluation configurations and functions for creating and previewing evaluations.

from reward_kit.evaluation import Evaluator, preview_evaluation, create_evaluation

Key components:

• Evaluator class: Manages metric loading, sample loading, and evaluator creation on the platform.
• preview_evaluation: Previews an evaluation with sample data before deployment.
• create_evaluation: Creates and deploys an evaluator to the platform.

​

config Module

The config module handles loading and managing configurations for the Reward Kit, typically from a rewardkit.yaml file.

from reward_kit.config import load_config, get_config, RewardKitConfig

Key functions and classes:

• load_config() / get_config(): Load the global Reward Kit configuration.
• RewardKitConfig: Pydantic model for the main configuration structure.
• Other models like GCPCloudRunConfig, AWSLambdaConfig.

​

models Module

The models module contains data models used throughout the Reward Kit.

from reward_kit.models import EvaluateResult, MetricResult, Message

​

rewards Module

The rewards module contains specialized reward functions for specific use cases.

from reward_kit.rewards.function_calling import match_function_call

​

server Module

The server module provides the RewardServer class and serve function to host reward functions as a FastAPI application.

from reward_kit.server import RewardServer, serve

Key components:

• RewardServer class: A class to encapsulate a reward function and run it as a server.
• serve() function: A utility to quickly serve a given reward function.

​

auth Module

The auth module provides utility functions to retrieve authentication credentials, primarily for Fireworks AI.

from reward_kit.auth import get_fireworks_api_key, get_fireworks_account_id

Key functions:

• get_fireworks_api_key(): Retrieves the Fireworks API key.
• get_fireworks_account_id(): Retrieves the Fireworks account ID.

​

gcp_tools Module

The gcp_tools module offers utilities for working with Google Cloud Platform, such as building and pushing Docker images to Artifact Registry and deploying to Cloud Run.

from reward_kit.gcp_tools import build_and_push_docker_image, deploy_to_cloud_run

​

packaging Module

The packaging module assists in preparing reward functions for deployment, for example, by generating Dockerfile content.

from reward_kit.packaging import generate_dockerfile_content

​

platform_api Module

The platform_api module provides functions for direct interaction with the Fireworks AI platform API, such as managing secrets.

from reward_kit.platform_api import create_or_update_fireworks_secret

​

rl_processing Module

The rl_processing module contains tools for processing data for Reinforcement Learning workflows, such as the RLDataAligner.

from reward_kit.rl_processing import RLDataAligner

​

mcp Module (reward_kit.mcp)

This sub-package contains components related to the Model Context Protocol (MCP).

• reward_kit.mcp.clients: Provides clients for interacting with MCP-compliant servers.

​

mcp_agent Module (reward_kit.mcp_agent)

This sub-package provides a framework for building and running agents that interact with MCP servers. It includes orchestration logic, various backend implementations, and a collection of pre-built MCP servers for common tasks (e.g., filesystem, git).

​

Command Line Interface

The Reward Kit provides a command-line interface for common operations:

# Show help
reward-kit --help

# Preview an evaluator
reward-kit preview --metrics-folders "metric=./path" --samples ./samples.jsonl

# Deploy an evaluator
reward-kit deploy --id my-evaluator --metrics-folders "metric=./path" --force

For detailed CLI documentation, see the CLI Reference.

​

Common Patterns

​

Creating a Basic Reward Function

from reward_kit import reward_function, EvaluateResult, MetricResult

@reward_function
def my_reward_function(messages, original_messages=None, **kwargs):
    # Your evaluation logic here
    response = messages[-1].get("content", "")
    # Assume calculate_score returns a float between 0.0 and 1.0
    # and calculate_success returns a boolean
    score = calculate_score(response)
    success = calculate_success(response) # Assume calculate_success is defined

    return EvaluateResult(
        score=score,
        reason="Overall evaluation reason for my_reward_function", # Added top-level reason
        metrics={
            "my_metric": MetricResult(
                score=score,
                success=success, # Added success field
                reason="Explanation for the metric score"
            )
        }
    )

​

Using a Deployed Reward Function

from reward_kit import RewardFunction

# Create a reference to a deployed reward function
reward_fn = RewardFunction(
    name="my-deployed-evaluator",
    mode="remote"
)

# Call the reward function
result = reward_fn(messages=[
    {"role": "user", "content": "What is machine learning?"},
    {"role": "assistant", "content": "Machine learning is..."}
])

print(f"Score: {result.score}")

​

Next Steps

• Explore the Examples for practical implementations
• Follow the Tutorials for step-by-step guidance
• Review the Developer Guide for conceptual understanding