Skip to content

Index

diapason.bench

Benchmarking framework for Diapason inference engines.

BaseBenchmark

Bases: ABC

Base class for all benchmark implementations.

Subclasses must be registered via @BenchmarkRegistry.register("name") to become discoverable.

description abstractmethod property

Human-readable description of what this benchmark measures.

name abstractmethod property

Short identifier for this benchmark.

run(engine, model, *, num_samples=10, **kwargs) abstractmethod

Execute the benchmark and return results.

BenchmarkRegistry

Bases: RegistryBase[Any]

Registry for benchmark implementations.

BenchmarkResult dataclass

Result from running a single benchmark.

BenchmarkSuite

Run a collection of benchmarks and aggregate results.

run_all(engine, model, *, num_samples=10, **kwargs)

Run all benchmarks and return a list of results.

summary(results)

Create a summary dict from benchmark results.

to_jsonl(results)

Serialize results to JSONL format (one JSON object per line).

ensure_registered()

Ensure all benchmark implementations are registered.