Skip to content

browser_assistant

diapason.evals.scorers.browser_assistant

browser_assistant scorer — exact match, semantic checklist, and source check.

Tier 1 (exact match): For exact facts, normalize and match numbers/names/versions. Tier 2 (semantic checklist): For semantic facts, binary yes/no via LLM judge. Tier 2 (quality checklist): No fabricated numbers, sources cited, answers question. Tier 1 (sources): Check if any URL or explicit reference is mentioned.

(exact/total_exact) * 0.35 + (semantic/total_semantic) * 0.35
  • quality_checklist * 0.15 + sources_cited * 0.15

BrowserAssistantScorer

Bases: Scorer

Score web research output by fact accuracy and sourcing.