flops
diapason.telemetry.flops
¶
FLOPs estimation and Model FLOPs Utilization (MFU) computation.
compute_mfu(flops, duration_s, gpu_name, num_gpus=1)
¶
Compute Model FLOPs Utilization.
MFU = actual_tflops / (peak_tflops * num_gpus)
estimate_flops(model, input_tokens, output_tokens)
¶
Estimate FLOPs for an inference pass (assumes KV caching).
Uses the 2 * P * T approximation where P = params, T = total tokens. Returns (total_flops, flops_per_token).
input_tokens must include system-prompt tokens and must not
be reduced for KV-cache reuse — it should represent the full prompt
size that was sent to the engine.
estimate_flops_no_kv_cache(model, input_tokens, output_tokens)
¶
Estimate FLOPs without KV caching (full recompute per token).
Without KV cache, each token is re-processed for every subsequent token. FLOPs = P * N * (N + 1) where P = params, N = total_tokens. Returns (total_flops, flops_per_token_avg).