Hugging Face· PlatformResearch paper

BigCodeBench: The Next Generation of HumanEval

Hugging Face announces the release of BigCodeBench, a new benchmark for evaluating LLMs on practical and challenging programming tasks without contamination, containing 1,140 function-level tasks from 139 libraries.

From the source

We are excited to announce the release of BigCodeBench, which evaluates LLMs on solving practical and challenging programming tasks without contamination. Specifically, BigCodeBench contains 1,140 function-level tasks to challenge LLMs to follow instructions and compose multiple function calls as tools from 139 libraries.

huggingface.co

Announced

18 Jun 2024 00:00 UTC

Detected

03 Aug 2026 21:12 UTC (+776d after)