grpo_trainer
diapason.learning.intelligence.orchestrator.grpo_trainer
¶
GRPO (Group Relative Policy Optimization) trainer for orchestrator.
Adapted from IPW's trainer.py. GRPO is simpler than PPO because it
doesn't require a separate critic model — instead, it uses
group-relative advantages: for each problem, sample N candidate
trajectories, compute rewards, normalise within the group, and update
the policy to increase the probability of better solutions.
All torch/transformers imports are guarded so the module can be
imported without GPU dependencies.