Skip to content

grpo_trainer

diapason.learning.intelligence.grpo_trainer

General-purpose GRPO trainer -- Group Relative Policy Optimization.

Fine-tunes any local model by sampling N responses per prompt, computing group-relative advantages, and applying a clipped policy gradient with KL penalty vs a frozen reference model.

DefaultRewardFn

Default reward function using length-normalized response quality heuristics.

score(prompt, response, ground_truth)

Score a response. Higher is better, range [0, 1].

GRPOTrainer

General-purpose GRPO trainer.

Parameters

config: GRPOConfig controlling model, sampling, and optimization params. reward_fn: Pluggable reward function. Defaults to DefaultRewardFn.

train(trace_store)

End-to-end: mine prompts from traces, then train.

Parameters

trace_store: Object with list_traces() returning trace objects.

train_on_prompts(prompts, ground_truths=None)

Run GRPO training on a set of prompts.

Parameters

prompts: List of prompt strings to train on. ground_truths: Optional parallel list of ground-truth answers for reward scoring.

RewardFn

Bases: Protocol

Protocol for reward functions used by GRPOTrainer.