Skip to content

Index

diapason.telemetry

Telemetry — SQLite-backed inference recording and instrumented wrappers.

AggregatedStats dataclass

Top-level summary combining per-model and per-engine stats.

BatchMetrics dataclass

Aggregated metrics for a batch of inference requests.

EfficiencyMetrics dataclass

Results of an MFU/MBU efficiency calculation.

EnergyBatch

Group inference requests into a batch and compute per-token energy.

Works with or without an EnergyMonitor. When no monitor is provided, request counts are still tracked but energy values stay at zero.

sample()

Wrap an energy monitor sample and provide a context for recording requests.

Yields a _BatchContext whose record_request() method should be called once per inference request inside the block.

EnergyMonitor

Bases: ABC

Abstract base class for energy measurement backends.

Each vendor implementation probes for hardware support at init, exposes an available() class method, and provides a sample() context manager that measures energy over a code block.

available() abstractmethod staticmethod

Return True if this monitor can run on the current hardware.

close() abstractmethod

Release any resources (handles, threads, etc.).

energy_method() abstractmethod

Return the measurement method: 'hw_counter', 'polling', 'rapl', or 'zeus'.

sample() abstractmethod

Context manager that measures energy during the enclosed block.

Yields an EnergySample that is populated when the block exits.

snapshot()

Return an instantaneous energy reading without start/stop bracket.

Subclasses should override to provide actual readings. Default returns an empty sample.

vendor() abstractmethod

Return the vendor enum for this monitor.

EnergySample dataclass

Aggregated energy metrics over an inference bracket.

Superset of GpuSample — adds vendor, device info, energy method, and per-component breakdown (CPU, GPU, DRAM, ANE).

EnergyVendor

Bases: str, Enum

Supported energy measurement vendors.

EngineStats dataclass

Aggregated statistics for a single engine backend.

GpuHardwareSpec dataclass

Peak theoretical capabilities for a known GPU model.

GpuMonitor

Background GPU poller using pynvml.

Usage::

mon = GpuMonitor(poll_interval_ms=50)
with mon.sample() as result:
    # ... run inference ...
    pass
print(result.energy_joules)
mon.close()

available() staticmethod

Return True if pynvml is importable and can be initialized.

close()

Shut down pynvml if it was initialized.

sample()

Context manager that polls GPUs during the block, then populates the sample.

If pynvml is unavailable or no devices are found, yields an empty :class:GpuSample without starting a background thread.

GpuSample dataclass

Aggregated GPU metrics over an inference bracket.

GpuSnapshot dataclass

A single point-in-time reading from one GPU device.

ModelStats dataclass

Aggregated statistics for a single model.

SteadyStateConfig dataclass

Configuration for steady-state detection.

SteadyStateDetector

Detect steady state using coefficient of variation over a sliding window.

The first warmup_samples recordings are always classified as warmup. After warmup, the CV (stdev / mean) of the last window_size values is checked. When CV < cv_threshold for min_steady_samples consecutive checks, steady state is declared.

result property

Return a snapshot of the detection state.

record(throughput, energy=0.0, latency=0.0)

Record a sample. Returns True when steady state is reached.

reset()

Clear all recorded state.

SteadyStateResult dataclass

Result of steady-state detection.

TelemetryAggregator

Read-only query layer over the telemetry SQLite database.

per_batch_stats(*, since=None, until=None, exclude_warmup=False)

Aggregate telemetry by batch_id.

Returns list of dicts with batch_id, total_requests, total_tokens, total_energy_joules, energy_per_token_joules.

TelemetrySample dataclass

Single telemetry sample.

TelemetrySession

Background-sampling telemetry session.

Spawns a daemon thread that calls monitor.snapshot() at the configured interval. Stores samples in a ring buffer (Rust-backed if available, else pure-Python fallback).

start()

Start background sampling thread.

stop()

Stop sampling thread.

TelemetryStore

Append-only SQLite store for inference telemetry records.

Writes are batched in memory and flushed to SQLite when a batch reaches batch_size, when flush_interval_seconds elapses (a background flusher thread guarantees this even with no further writes), on any read through this store, and on close(). Readers that open their OWN connection to the database file (e.g. TelemetryAggregator) therefore see new rows within flush_interval_seconds at the latest; call flush() first for immediate visibility. Pass flush_interval_seconds=0 to disable time-based flushing (batch-size and read/close flushes still apply).

close()

Flush pending records and close the underlying SQLite connection.

flush()

Write all pending records to the database.

list_recent(limit=50)

Return recent telemetry rows as dictionaries.

list_recent_mining_stats(limit=50)

Return recent mining stats snapshots as dictionaries.

record(rec)

Persist a single telemetry record.

record_mining_stats(stats)

Persist one mining stats snapshot.

stats is duck-typed to keep telemetry usable without importing the optional mining package at module import time.

subscribe_to_bus(bus)

Subscribe to TELEMETRY_RECORD events on bus.

VLLMMetrics dataclass

Parsed vLLM performance metrics.

VLLMMetricsScraper

Scrapes vLLM's Prometheus /metrics endpoint.

scrape()

Fetch and parse vLLM metrics. Returns zeroed metrics on error.

compute_efficiency(param_count_b, active_params_b, gpu_peak_tflops, gpu_peak_bandwidth_gb_s, tokens_per_sec, num_gpus=1, energy_joules=0.0, accuracy=0.0, bytes_per_param=2.0)

Compute MFU, MBU, and derived efficiency metrics.

Parameters:

Name Type Description Default
param_count_b float

Total parameter count in billions.

required
active_params_b float | None

Active parameters per token in billions (None for dense).

required
gpu_peak_tflops float

Peak theoretical TFLOPS per GPU (e.g. 312 for A100 SXM FP16).

required
gpu_peak_bandwidth_gb_s float

Peak memory bandwidth per GPU in GB/s (e.g. 2039 for A100 SXM).

required
tokens_per_sec float

Measured generation throughput (tokens/second).

required
num_gpus int

Number of GPUs used for inference.

1
energy_joules float

Total energy consumed in joules (for IPJ calculation).

0.0
accuracy float

Accuracy score in [0, 1] (for IPJ calculation).

0.0
bytes_per_param float

Bytes per parameter (default 2.0 for FP16).

2.0

Returns:

Type Description
EfficiencyMetrics

class:EfficiencyMetrics with all computed values.

compute_itl_stats(token_timestamps)

Compute ITL statistics from token arrival timestamps (in ms).

Returns dict with p50_ms, p90_ms, p95_ms, p99_ms, mean_ms, min_ms, max_ms.

compute_mfu(flops, duration_s, gpu_name, num_gpus=1)

Compute Model FLOPs Utilization.

MFU = actual_tflops / (peak_tflops * num_gpus)

compute_phase_metrics(session, start_ns, end_ns, tokens)

Compute energy/power metrics for a phase window.

create_energy_monitor(poll_interval_ms=50, prefer_vendor=None)

Factory — auto-detect and return the best available EnergyMonitor.

Detection order: NVIDIA > AMD > Apple > CPU RAPL. If prefer_vendor is set, try that vendor first.

Returns None if no energy monitoring is available.

estimate_flops(model, input_tokens, output_tokens)

Estimate FLOPs for an inference pass (assumes KV caching).

Uses the 2 * P * T approximation where P = params, T = total tokens. Returns (total_flops, flops_per_token).

input_tokens must include system-prompt tokens and must not be reduced for KV-cache reuse — it should represent the full prompt size that was sent to the engine.

estimate_flops_no_kv_cache(model, input_tokens, output_tokens)

Estimate FLOPs without KV caching (full recompute per token).

Without KV cache, each token is re-processed for every subsequent token. FLOPs = P * N * (N + 1) where P = params, N = total_tokens. Returns (total_flops, flops_per_token_avg).

instrumented_generate(engine, messages, *, model, bus, temperature=0.7, max_tokens=1024, **kwargs)

Call engine.generate() and publish telemetry events on bus.

Returns the raw result dict from the engine.

split_at_ttft(session, start_ns, ttft_ns, end_ns, input_tokens, output_tokens)

Split energy at TTFT boundary into prefill and decode phases.