Skip to content

pinchbench_env

diapason.evals.execution.pinchbench_env

PinchBench task environment — per-task workspace setup and grading.

PinchBenchTaskEnv

Per-task workspace environment for PinchBench.

Context manager that creates an isolated workspace directory, populates fixture files from the task definition, and runs grading after agent execution via run_tests().

run_tests()

Grade the agent's work using PinchBench grading logic.

Called by AgenticRunner after agent execution, before QueryTrace is constructed. Builds transcript from raw EventRecorder events.

set_event_recorder(recorder)

Receive the EventRecorder from AgenticRunner for transcript building.