Skip to content

terminalbench_judge

diapason.evals.scorers.terminalbench_judge

TerminalBench scorer — LLM-as-judge for terminal task evaluation.

Compares predicted terminal output / commands against the expected answer using an LLM judge.

TerminalBenchScorer

Bases: LLMJudgeScorer

LLM-as-judge evaluation for TerminalBench terminal tasks.