Skip to content

tracker

diapason.evals.core.tracker

ResultTracker ABC for external experiment tracking.

ResultTracker

Bases: ABC

Abstract base class for experiment result trackers.

Lifecycle: on_run_start -> on_result (per sample) -> on_summary -> on_run_end.

on_result(result, config) abstractmethod

Called after each sample is evaluated.

on_run_end() abstractmethod

Called at the very end of a run for cleanup.

on_run_start(config) abstractmethod

Called once before evaluation begins.

on_summary(summary) abstractmethod

Called after all samples are evaluated with aggregate stats.