tracker
diapason.evals.core.tracker
¶
ResultTracker ABC for external experiment tracking.
ResultTracker
¶
Bases: ABC
Abstract base class for experiment result trackers.
Lifecycle: on_run_start -> on_result (per sample) -> on_summary -> on_run_end.
on_result(result, config)
abstractmethod
¶
Called after each sample is evaluated.
on_run_end()
abstractmethod
¶
Called at the very end of a run for cleanup.
on_run_start(config)
abstractmethod
¶
Called once before evaluation begins.
on_summary(summary)
abstractmethod
¶
Called after all samples are evaluated with aggregate stats.