Skip to content

eval_cmd

diapason.cli.eval_cmd

diapason eval — evaluation framework CLI commands.

eval_compare(result_files, metric)

Compare results from multiple eval runs.

eval_group()

Evaluation framework — benchmark models, agents, and learning.

eval_list()

List available benchmarks and backends.

eval_report(result_file)

Generate detailed report from eval results.

eval_run(config_path, benchmark, model, max_samples, backend, base_url, api_key, agent_name, engine_key, tools, telemetry, gpu_metrics, seed, temperature, max_tokens, model_filter, output_path, wandb_project, wandb_entity, wandb_tags, wandb_group, sheets_spreadsheet_id, sheets_worksheet, sheets_credentials_path, verbose)

Run evaluation benchmarks.