Index
diapason.telemetry
¶
Telemetry — SQLite-backed inference recording and instrumented wrappers.
AggregatedStats
dataclass
¶
Top-level summary combining per-model and per-engine stats.
BatchMetrics
dataclass
¶
Aggregated metrics for a batch of inference requests.
EfficiencyMetrics
dataclass
¶
Results of an MFU/MBU efficiency calculation.
EnergyBatch
¶
Group inference requests into a batch and compute per-token energy.
Works with or without an EnergyMonitor. When no monitor is provided,
request counts are still tracked but energy values stay at zero.
sample()
¶
Wrap an energy monitor sample and provide a context for recording requests.
Yields a _BatchContext whose record_request() method should be
called once per inference request inside the block.
EnergyMonitor
¶
Bases: ABC
Abstract base class for energy measurement backends.
Each vendor implementation probes for hardware support at init,
exposes an available() class method, and provides a sample()
context manager that measures energy over a code block.
available()
abstractmethod
staticmethod
¶
Return True if this monitor can run on the current hardware.
close()
abstractmethod
¶
Release any resources (handles, threads, etc.).
energy_method()
abstractmethod
¶
Return the measurement method: 'hw_counter', 'polling', 'rapl', or 'zeus'.
sample()
abstractmethod
¶
Context manager that measures energy during the enclosed block.
Yields an EnergySample that is populated when the block exits.
snapshot()
¶
Return an instantaneous energy reading without start/stop bracket.
Subclasses should override to provide actual readings. Default returns an empty sample.
vendor()
abstractmethod
¶
Return the vendor enum for this monitor.
EnergySample
dataclass
¶
Aggregated energy metrics over an inference bracket.
Superset of GpuSample — adds vendor, device info, energy method,
and per-component breakdown (CPU, GPU, DRAM, ANE).
EnergyVendor
¶
Bases: str, Enum
Supported energy measurement vendors.
EngineStats
dataclass
¶
Aggregated statistics for a single engine backend.
GpuHardwareSpec
dataclass
¶
Peak theoretical capabilities for a known GPU model.
GpuMonitor
¶
Background GPU poller using pynvml.
Usage::
mon = GpuMonitor(poll_interval_ms=50)
with mon.sample() as result:
# ... run inference ...
pass
print(result.energy_joules)
mon.close()
available()
staticmethod
¶
Return True if pynvml is importable and can be initialized.
close()
¶
Shut down pynvml if it was initialized.
sample()
¶
Context manager that polls GPUs during the block, then populates the sample.
If pynvml is unavailable or no devices are found, yields an empty
:class:GpuSample without starting a background thread.
GpuSample
dataclass
¶
Aggregated GPU metrics over an inference bracket.
GpuSnapshot
dataclass
¶
A single point-in-time reading from one GPU device.
ModelStats
dataclass
¶
Aggregated statistics for a single model.
SteadyStateConfig
dataclass
¶
Configuration for steady-state detection.
SteadyStateDetector
¶
Detect steady state using coefficient of variation over a sliding window.
The first warmup_samples recordings are always classified as warmup.
After warmup, the CV (stdev / mean) of the last window_size values is
checked. When CV < cv_threshold for min_steady_samples consecutive
checks, steady state is declared.
SteadyStateResult
dataclass
¶
Result of steady-state detection.
TelemetryAggregator
¶
Read-only query layer over the telemetry SQLite database.
per_batch_stats(*, since=None, until=None, exclude_warmup=False)
¶
Aggregate telemetry by batch_id.
Returns list of dicts with batch_id, total_requests, total_tokens, total_energy_joules, energy_per_token_joules.
TelemetrySample
dataclass
¶
Single telemetry sample.
TelemetrySession
¶
TelemetryStore
¶
Append-only SQLite store for inference telemetry records.
Writes are batched in memory and flushed to SQLite when a batch reaches
batch_size, when flush_interval_seconds elapses (a background
flusher thread guarantees this even with no further writes), on any read
through this store, and on close(). Readers that open their OWN
connection to the database file (e.g. TelemetryAggregator) therefore
see new rows within flush_interval_seconds at the latest; call
flush() first for immediate visibility. Pass
flush_interval_seconds=0 to disable time-based flushing (batch-size
and read/close flushes still apply).
close()
¶
Flush pending records and close the underlying SQLite connection.
flush()
¶
Write all pending records to the database.
list_recent(limit=50)
¶
Return recent telemetry rows as dictionaries.
list_recent_mining_stats(limit=50)
¶
Return recent mining stats snapshots as dictionaries.
record(rec)
¶
Persist a single telemetry record.
record_mining_stats(stats)
¶
Persist one mining stats snapshot.
stats is duck-typed to keep telemetry usable without importing the
optional mining package at module import time.
subscribe_to_bus(bus)
¶
Subscribe to TELEMETRY_RECORD events on bus.
VLLMMetrics
dataclass
¶
Parsed vLLM performance metrics.
VLLMMetricsScraper
¶
Scrapes vLLM's Prometheus /metrics endpoint.
scrape()
¶
Fetch and parse vLLM metrics. Returns zeroed metrics on error.
compute_efficiency(param_count_b, active_params_b, gpu_peak_tflops, gpu_peak_bandwidth_gb_s, tokens_per_sec, num_gpus=1, energy_joules=0.0, accuracy=0.0, bytes_per_param=2.0)
¶
Compute MFU, MBU, and derived efficiency metrics.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
param_count_b
|
float
|
Total parameter count in billions. |
required |
active_params_b
|
float | None
|
Active parameters per token in billions (None for dense). |
required |
gpu_peak_tflops
|
float
|
Peak theoretical TFLOPS per GPU (e.g. 312 for A100 SXM FP16). |
required |
gpu_peak_bandwidth_gb_s
|
float
|
Peak memory bandwidth per GPU in GB/s (e.g. 2039 for A100 SXM). |
required |
tokens_per_sec
|
float
|
Measured generation throughput (tokens/second). |
required |
num_gpus
|
int
|
Number of GPUs used for inference. |
1
|
energy_joules
|
float
|
Total energy consumed in joules (for IPJ calculation). |
0.0
|
accuracy
|
float
|
Accuracy score in [0, 1] (for IPJ calculation). |
0.0
|
bytes_per_param
|
float
|
Bytes per parameter (default 2.0 for FP16). |
2.0
|
Returns:
| Type | Description |
|---|---|
EfficiencyMetrics
|
class: |
compute_itl_stats(token_timestamps)
¶
Compute ITL statistics from token arrival timestamps (in ms).
Returns dict with p50_ms, p90_ms, p95_ms, p99_ms, mean_ms, min_ms, max_ms.
compute_mfu(flops, duration_s, gpu_name, num_gpus=1)
¶
Compute Model FLOPs Utilization.
MFU = actual_tflops / (peak_tflops * num_gpus)
compute_phase_metrics(session, start_ns, end_ns, tokens)
¶
Compute energy/power metrics for a phase window.
create_energy_monitor(poll_interval_ms=50, prefer_vendor=None)
¶
Factory — auto-detect and return the best available EnergyMonitor.
Detection order: NVIDIA > AMD > Apple > CPU RAPL. If prefer_vendor is set, try that vendor first.
Returns None if no energy monitoring is available.
estimate_flops(model, input_tokens, output_tokens)
¶
Estimate FLOPs for an inference pass (assumes KV caching).
Uses the 2 * P * T approximation where P = params, T = total tokens. Returns (total_flops, flops_per_token).
input_tokens must include system-prompt tokens and must not
be reduced for KV-cache reuse — it should represent the full prompt
size that was sent to the engine.
estimate_flops_no_kv_cache(model, input_tokens, output_tokens)
¶
Estimate FLOPs without KV caching (full recompute per token).
Without KV cache, each token is re-processed for every subsequent token. FLOPs = P * N * (N + 1) where P = params, N = total_tokens. Returns (total_flops, flops_per_token_avg).
instrumented_generate(engine, messages, *, model, bus, temperature=0.7, max_tokens=1024, **kwargs)
¶
Call engine.generate() and publish telemetry events on bus.
Returns the raw result dict from the engine.
split_at_ttft(session, start_ns, ttft_ns, end_ns, input_tokens, output_tokens)
¶
Split energy at TTFT boundary into prefill and decode phases.