Skip to content

batch

diapason.telemetry.batch

Batch-level energy accounting — group requests and compute per-token energy.

BatchMetrics dataclass

Aggregated metrics for a batch of inference requests.

EnergyBatch

Group inference requests into a batch and compute per-token energy.

Works with or without an EnergyMonitor. When no monitor is provided, request counts are still tracked but energy values stay at zero.

sample()

Wrap an energy monitor sample and provide a context for recording requests.

Yields a _BatchContext whose record_request() method should be called once per inference request inside the block.