Hugging Face· PlatformResearch paper

Alyah ⭐️: Toward Robust Evaluation of Emirati Dialect Capabilities in Arabic LLMs

Hugging Face published a community article introducing Alyah, a benchmark for evaluating Arabic LLMs on the Emirati dialect. The benchmark contains 1,173 manually collected multiple-choice samples covering greetings, figurative language, poetry, and heritage. 54 models (23 base, 31 instruction-tuned) were evaluated, including Jais, Allam, Qwen, LLaMA, Fanar, and AceGPT. Accuracy on multiple-choice questions is the primary metric.

From the source

our team introduces Alyah (which means North Star ⭐️ in Emirati), an Emirati-centric benchmark designed to assess how well Arabic LLMs capture the linguistic, cultural, and pragmatic aspects of the Emirati dialect.

huggingface.co

Announced

27 Jan 2026 10:26 UTC

Detected