scorer
diapason.learning.optimize.personal.scorer
¶
LLM-judge scorer for personal benchmarks.
PersonalBenchmarkScorer
¶
Bases: LLMJudgeScorer
Judges a candidate response against the best-known response from traces.
score(record, model_answer)
¶
Compare model_answer against record.reference using the judge LLM.
Returns (is_correct, metadata) where is_correct indicates whether
the candidate answer is at least as good as the reference.