# Hugging Face — BigCodeBench: The Next Generation of HumanEval

- Company: Hugging Face (huggingface.co)
- Announced: 2024-06-18T00:00:00+00:00
- Category: research-paper
- Subject: Platform
- Source: https://huggingface.co/blog/leaderboard-bigcodebench
- Record: https://forck.live/items/1870-bigcodebench-the-next-generation-of-humaneval

Hugging Face announces the release of BigCodeBench, a new benchmark for evaluating LLMs on practical and challenging programming tasks without contamination, containing 1,140 function-level tasks from 139 libraries.

## Evidence

Verbatim from https://huggingface.co/blog/leaderboard-bigcodebench:

> We are excited to announce the release of BigCodeBench, which evaluates LLMs on solving practical and challenging programming tasks without contamination. Specifically, BigCodeBench contains 1,140 function-level tasks to challenge LLMs to follow instructions and compose multiple function calls as tools from 139 libraries.

---

Record: https://forck.live/items/1870-bigcodebench-the-next-generation-of-humaneval
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md