eval_cmd
diapason.cli.eval_cmd
¶
diapason eval — evaluation framework CLI commands.
eval_compare(result_files, metric)
¶
Compare results from multiple eval runs.
eval_group()
¶
Evaluation framework — benchmark models, agents, and learning.
eval_list()
¶
List available benchmarks and backends.
eval_report(result_file)
¶
Generate detailed report from eval results.
eval_run(config_path, benchmark, model, max_samples, backend, base_url, api_key, agent_name, engine_key, tools, telemetry, gpu_metrics, seed, temperature, max_tokens, model_filter, output_path, wandb_project, wandb_entity, wandb_tags, wandb_group, sheets_spreadsheet_id, sheets_worksheet, sheets_credentials_path, verbose)
¶
Run evaluation benchmarks.