📚 3LM: A Benchmark for Arabic LLMs in STEM and Code

Hugging Face announces the release of 3LM, a benchmark for evaluating Arabic large language models on STEM and code generation, consisting of three datasets: Native STEM, Synthetic STEM, and translated code benchmarks (HumanEval-ar, MBPP-ar). The post reports evaluation results for over 40 models.

From the source
To address this gap, we introduce 3LM (علم), a multi-component benchmark tailored to evaluate Arabic LLMs on STEM (Science, Technology, Engineering, and Mathematics) subjects and code generation.
huggingface.co

Visit source ↗huggingface.co

Announced: 01 Aug 2025 14:25 UTC
Detected: 03 Aug 2026 21:12 UTC (+367d after)

Other stories

Hugging Face · 28 AugThe Open ASR Leaderboard Adds Its First Global South LanguageCapability change

Google Research · 27 AugPlanetary prediction engine: Automating global models via Earth AIResearch paper

OpenAI · 27 AugBetter answers, broader thinking: What students gain from ChatGPT and critical-thinking trainingResearch paper

LG AI Research · 27 Aug

[ACL 2026] ReSQL: A Self-Improving Text-to-SQL Framework through Retrieval-Augmented Error Reasoning

Research paper

OpenAI · 31 AugA milestone in expanding access to AICompany news

xAI · 29 AugGrok Bot now works with XCapability change