Skip to content

reward

diapason.learning.intelligence.orchestrator.reward

Multi-objective reward function for orchestrator training.

Adapted from IPW's reward.py. Balances accuracy, cost, energy, latency, and power into a single scalar reward used by both the SFT grading pipeline and the GRPO policy gradient.

AdaptiveRewardWeights

Adaptive reward weights that shift during training.

Early training focuses on accuracy (higher alpha). Late training optimises efficiency (higher cost/energy/power weights).

get_weights(current_step)

Get weights for current_step via linear interpolation.

MultiObjectiveReward

Multi-objective reward combining accuracy, cost, energy, latency, power.

Formula::

reward = alpha * accuracy
         - beta_cost  * (cost   / cost_scale)
         - beta_energy * (energy / energy_scale)
         - gamma_latency * (latency / latency_scale)
         - gamma_power   * (power   / power_scale)

compute(episode)

Compute scalar reward for an episode.

compute_batch(episodes)

Compute rewards for a batch of episodes.

compute_with_breakdown(episode)

Compute reward with detailed per-component breakdown.

Normalizers dataclass

Normalization constants for reward scaling.

These are typical values used to scale metrics to similar ranges. Tune based on your specific tools and tasks.

RewardWeights dataclass

Weights for multi-objective reward function.

Each metric has its own coefficient: - alpha: Accuracy (correctness of answer) - beta_cost: API/cloud cost in USD - beta_energy: Energy consumption in joules - gamma_latency: Response time in seconds - gamma_power: Peak power usage in watts

__post_init__()

Validate weights sum to 1.0.