Skip to content

frames_judge

frames_judge

FRAMES scorer — LLM-as-judge for multi-hop factual retrieval.

Adapted from IPW's frames.py evaluation handler.

Classes

FRAMESScorer 

FRAMESScorer(
    judge_backend: InferenceBackend, judge_model: str
)

Bases: LLMJudgeScorer

LLM-as-judge evaluation for FRAMES multi-hop factual retrieval.

Source code in src/diapason/evals/core/scorer.py 
33
34
35
def __init__(self, judge_backend: InferenceBackend, judge_model: str) -> None:
    self._judge_backend = judge_backend
    self._judge_model = judge_model