livecodebench
diapason.evals.datasets.livecodebench
¶
LiveCodeBench dataset provider — competitive programming code generation.
Loads problems from the LiveCodeBench HuggingFace dataset (livecodebench/code_generation_lite) for evaluating code generation capability.
https://livecodebench.github.io/
https://github.com/LiveCodeBench/LiveCodeBench
LiveCodeBenchDataset
¶
Bases: DatasetProvider
LiveCodeBench competitive programming benchmark.
Loads problems from the LiveCodeBench HuggingFace dataset for evaluating code generation capability on fresh competitive programming problems from LeetCode, AtCoder, and CodeForces.