# Hugging Face — Alyah ⭐️: Toward Robust Evaluation of Emirati Dialect Capabilities in Arabic LLMs

- Company: Hugging Face (huggingface.co)
- Announced: 2026-01-27T10:26:42+00:00
- Category: research-paper
- Subject: Platform
- Source: https://huggingface.co/blog/tiiuae/emirati-benchmarks
- Record: https://forck.live/items/1558-alyah-toward-robust-evaluation-of-emirati-dialect-capabilities-in-arabic-llms

Hugging Face published a community article introducing Alyah, a benchmark for evaluating Arabic LLMs on the Emirati dialect. The benchmark contains 1,173 manually collected multiple-choice samples covering greetings, figurative language, poetry, and heritage. 54 models (23 base, 31 instruction-tuned) were evaluated, including Jais, Allam, Qwen, LLaMA, Fanar, and AceGPT. Accuracy on multiple-choice questions is the primary metric.

## Evidence

Verbatim from https://huggingface.co/blog/tiiuae/emirati-benchmarks:

> our team introduces Alyah (which means North Star ⭐️ in Emirati), an Emirati-centric benchmark designed to assess how well Arabic LLMs capture the linguistic, cultural, and pragmatic aspects of the Emirati dialect.

---

Record: https://forck.live/items/1558-alyah-toward-robust-evaluation-of-emirati-dialect-capabilities-in-arabic-llms
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md