Skip to content

scorer

diapason.learning.optimize.personal.scorer

LLM-judge scorer for personal benchmarks.

PersonalBenchmarkScorer

Bases: LLMJudgeScorer

Judges a candidate response against the best-known response from traces.

score(record, model_answer)

Compare model_answer against record.reference using the judge LLM.

Returns (is_correct, metadata) where is_correct indicates whether the candidate answer is at least as good as the reference.