Skip to content

knowledge_base

knowledge_base

Knowledge base scorer — answer correctness via normalized match + LLM fallback.

Evaluates document-grounded QA by checking if the model answer matches the reference answer using exact match with normalization, falling back to LLM judge for semantic comparison.

Classes

KnowledgeBaseScorer 

KnowledgeBaseScorer(
    judge_backend: InferenceBackend, judge_model: str
)

Bases: LLMJudgeScorer

Score knowledge base QA: answer correctness.

Source code in src/diapason/evals/core/scorer.py 
33
34
35
def __init__(self, judge_backend: InferenceBackend, judge_model: str) -> None:
    self._judge_backend = judge_backend
    self._judge_model = judge_model