browser_assistant
diapason.evals.scorers.browser_assistant
¶
browser_assistant scorer — exact match, semantic checklist, and source check.
Tier 1 (exact match): For exact facts, normalize and match numbers/names/versions. Tier 2 (semantic checklist): For semantic facts, binary yes/no via LLM judge. Tier 2 (quality checklist): No fabricated numbers, sources cited, answers question. Tier 1 (sources): Check if any URL or explicit reference is mentioned.
(exact/total_exact) * 0.35 + (semantic/total_semantic) * 0.35
- quality_checklist * 0.15 + sources_cited * 0.15