Skip to content

grpo_trainer

diapason.learning.intelligence.orchestrator.grpo_trainer

GRPO (Group Relative Policy Optimization) trainer for orchestrator.

Adapted from IPW's trainer.py. GRPO is simpler than PPO because it doesn't require a separate critic model — instead, it uses group-relative advantages: for each problem, sample N candidate trajectories, compute rewards, normalise within the group, and update the policy to increase the probability of better solutions.

All torch/transformers imports are guarded so the module can be imported without GPU dependencies.

OrchestratorGRPOConfig dataclass

Configuration for orchestrator GRPO training.

OrchestratorGRPOTrainer

GRPO trainer for orchestrator policy.

torch must be installed to call :meth:train.

train()

Run the GRPO training loop.