# Hugging Face — Open LLM Leaderboard: DROP deep dive

- Company: Hugging Face (huggingface.co)
- Announced: 2023-12-01
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://huggingface.co/blog/open-llm-leaderboard-drop
- Record: https://forck.live/items/1972-open-llm-leaderboard-drop-deep-dive
- Subject: Platform

This blog post investigates unexpectedly low DROP benchmark scores on the Open LLM Leaderboard, identifying issues with normalization and the use of '.' as an end-of-generation token, and proposes using '\n' instead.

## Evidence

Verbatim from https://huggingface.co/blog/open-llm-leaderboard-drop:

> We did a deep dive to understand what was going on, come with us to see what we found out!

---

Record: https://forck.live/items/1972-open-llm-leaderboard-drop-deep-dive
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md