Skip to content

supergpqa_mcq

diapason.evals.scorers.supergpqa_mcq

SuperGPQA MCQ scorer — LLM-based letter extraction + exact match.

Adapted from IPW's mcq.py and gpqa.py evaluation handlers.

SuperGPQAScorer

Bases: LLMJudgeScorer

Score SuperGPQA responses by extracting answer letter via LLM.