From the source
Hugging Face and Adyen introduce DABStep, a benchmark of over 450 real-world data analysis tasks for evaluating LLM agents.
Current models achieve only 16% accuracy.
From the source
Adyen and Hugging Face built the Data Agent Benchmark for Multi-step Reasoning (DABstep) together. DABstep consists of over 450 data analysis tasks designed to evaluate the capabilities of state-of-the-art LLMs and AI agents. Our findings reveal that DABstep presents a significant challenge for current AI models, with the most capable reasoning-based agents achieving only 16% accuracy, highlighting significant progress to be made in the field.
huggingface.co
