Skip to content

efficiency

diapason.telemetry.efficiency

MFU/MBU efficiency calculator for GPU inference telemetry.

Computes Model FLOPs Utilization (MFU) and Model Bandwidth Utilization (MBU) to quantify how efficiently a model uses available GPU compute and memory bandwidth.

EfficiencyMetrics dataclass

Results of an MFU/MBU efficiency calculation.

compute_efficiency(param_count_b, active_params_b, gpu_peak_tflops, gpu_peak_bandwidth_gb_s, tokens_per_sec, num_gpus=1, energy_joules=0.0, accuracy=0.0, bytes_per_param=2.0)

Compute MFU, MBU, and derived efficiency metrics.

Parameters:

Name Type Description Default
param_count_b float

Total parameter count in billions.

required
active_params_b float | None

Active parameters per token in billions (None for dense).

required
gpu_peak_tflops float

Peak theoretical TFLOPS per GPU (e.g. 312 for A100 SXM FP16).

required
gpu_peak_bandwidth_gb_s float

Peak memory bandwidth per GPU in GB/s (e.g. 2039 for A100 SXM).

required
tokens_per_sec float

Measured generation throughput (tokens/second).

required
num_gpus int

Number of GPUs used for inference.

1
energy_joules float

Total energy consumed in joules (for IPJ calculation).

0.0
accuracy float

Accuracy score in [0, 1] (for IPJ calculation).

0.0
bytes_per_param float

Bytes per parameter (default 2.0 for FP16).

2.0

Returns:

Type Description
EfficiencyMetrics

class:EfficiencyMetrics with all computed values.

estimate_model_bytes_per_token(param_count_b, bytes_per_param=2.0)

Estimate bytes of memory loaded per decode step.

Parameters:

Name Type Description Default
param_count_b float

Total parameter count in billions.

required
bytes_per_param float

Bytes per parameter (default 2.0 for FP16).

2.0

Returns:

Type Description
float

Bytes loaded per token.

estimate_model_flops_per_token(param_count_b, active_params_b=None)

Estimate FLOPs for one forward-pass token of a dense transformer.

For dense models, FLOPs per token ≈ 2 * params. For MoE models, pass active_params_b (the number of active parameters per token).

Parameters:

Name Type Description Default
param_count_b float

Total parameter count in billions.

required
active_params_b float | None

Active parameters per token in billions. If None, defaults to param_count_b (dense model).

None

Returns:

Type Description
float

Estimated FLOPs per token.