Skip to content

taubench

diapason.evals.datasets.taubench

TauBench V2 dataset provider — multi-turn customer service benchmark.

Wraps the tau2-bench framework for evaluation within Diapason. Supports airline, retail, and telecom domains.

Reference: https://github.com/sierra-research/tau2-bench

TauBenchDataset

Bases: DatasetProvider

TauBench V2 multi-turn customer service benchmark.

Wraps tau2-bench's task loading and evaluation infrastructure. Each EvalRecord represents a single customer service scenario.

create_task_env(record)

Create a TauBench task environment for evaluation.

set_engine_config(engine_key=None, model=None, temperature=0.7, max_tokens=4096, user_model=None, num_trials=None, telemetry=False, gpu_metrics=False)

Inject engine configuration for the agent. Called by CLI.