simpleqa_judge
diapason.evals.scorers.simpleqa_judge
¶
SimpleQA scorer -- normalized exact match with LLM fallback.
Evaluates short factual answers using exact string matching (with normalization) and falls back to an LLM judge for semantic comparison.