reward
diapason.learning.intelligence.orchestrator.reward
¶
Multi-objective reward function for orchestrator training.
Adapted from IPW's reward.py. Balances accuracy, cost, energy,
latency, and power into a single scalar reward used by both the SFT
grading pipeline and the GRPO policy gradient.
AdaptiveRewardWeights
¶
Adaptive reward weights that shift during training.
Early training focuses on accuracy (higher alpha). Late training optimises efficiency (higher cost/energy/power weights).
get_weights(current_step)
¶
Get weights for current_step via linear interpolation.
MultiObjectiveReward
¶
Multi-objective reward combining accuracy, cost, energy, latency, power.
Formula::
reward = alpha * accuracy
- beta_cost * (cost / cost_scale)
- beta_energy * (energy / energy_scale)
- gamma_latency * (latency / latency_scale)
- gamma_power * (power / power_scale)
Normalizers
dataclass
¶
Normalization constants for reward scaling.
These are typical values used to scale metrics to similar ranges. Tune based on your specific tools and tasks.
RewardWeights
dataclass
¶
Weights for multi-objective reward function.
Each metric has its own coefficient: - alpha: Accuracy (correctness of answer) - beta_cost: API/cloud cost in USD - beta_energy: Energy consumption in joules - gamma_latency: Response time in seconds - gamma_power: Peak power usage in watts
__post_init__()
¶
Validate weights sum to 1.0.