Skip to content

livecodebench

diapason.evals.datasets.livecodebench

LiveCodeBench dataset provider — competitive programming code generation.

Loads problems from the LiveCodeBench HuggingFace dataset (livecodebench/code_generation_lite) for evaluating code generation capability.

https://livecodebench.github.io/

https://github.com/LiveCodeBench/LiveCodeBench

LiveCodeBenchDataset

Bases: DatasetProvider

LiveCodeBench competitive programming benchmark.

Loads problems from the LiveCodeBench HuggingFace dataset for evaluating code generation capability on fresh competitive programming problems from LeetCode, AtCoder, and CodeForces.