grpo_trainer
diapason.learning.intelligence.grpo_trainer
¶
General-purpose GRPO trainer -- Group Relative Policy Optimization.
Fine-tunes any local model by sampling N responses per prompt, computing group-relative advantages, and applying a clipped policy gradient with KL penalty vs a frozen reference model.
DefaultRewardFn
¶
Default reward function using length-normalized response quality heuristics.
score(prompt, response, ground_truth)
¶
Score a response. Higher is better, range [0, 1].
GRPOTrainer
¶
General-purpose GRPO trainer.
Parameters¶
config:
GRPOConfig controlling model, sampling, and optimization params.
reward_fn:
Pluggable reward function. Defaults to DefaultRewardFn.
RewardFn
¶
Bases: Protocol
Protocol for reward functions used by GRPOTrainer.