Skip to content

flops

diapason.telemetry.flops

FLOPs estimation and Model FLOPs Utilization (MFU) computation.

compute_mfu(flops, duration_s, gpu_name, num_gpus=1)

Compute Model FLOPs Utilization.

MFU = actual_tflops / (peak_tflops * num_gpus)

estimate_flops(model, input_tokens, output_tokens)

Estimate FLOPs for an inference pass (assumes KV caching).

Uses the 2 * P * T approximation where P = params, T = total tokens. Returns (total_flops, flops_per_token).

input_tokens must include system-prompt tokens and must not be reduced for KV-cache reuse — it should represent the full prompt size that was sent to the engine.

estimate_flops_no_kv_cache(model, input_tokens, output_tokens)

Estimate FLOPs without KV caching (full recompute per token).

Without KV cache, each token is re-processed for every subsequent token. FLOPs = P * N * (N + 1) where P = params, N = total_tokens. Returns (total_flops, flops_per_token_avg).