Skip to content

terminalbench_native

diapason.evals.datasets.terminalbench_native

TerminalBench Native dataset — loads from the terminal-bench pip package (v2 API).

Agentic benchmark using the native terminal-bench SDK for task loading and test-based evaluation.

TerminalBenchNativeDataset

Bases: DatasetProvider

TerminalBench using the native terminal-bench pip package (v2 API).

create_task_env(record)

Return a TerminalBenchTaskEnv for the given record.

verify_requirements()

Check that terminal-bench and docker are available.