A New Framework for Evaluating Voice Agents (EVA)

ServiceNow AI introduces EVA, an end-to-end evaluation framework for conversational voice agents that jointly scores task accuracy (EVA-A) and conversational experience (EVA-X). They release an initial airline dataset of 50 scenarios and provide benchmark results for 20 systems, finding a consistent Accuracy-Experience tradeoff.

From the source
We introduce EVA, an end-to-end evaluation framework for conversational voice agents that evaluates complete, multi-turn spoken conversations using a realistic bot-to-bot architecture. EVA produces two high-level scores, EVA-A (Accuracy) and EVA-X (Experience), and is designed to surface failures along each dimension.
huggingface.co

Visit source ↗huggingface.co

Announced: 24 Mar 2026 02:01 UTC
Detected: 03 Aug 2026 21:12 UTC (+132d after)

Other stories

Hugging Face · 28 AugThe Open ASR Leaderboard Adds Its First Global South LanguageCapability change

Google Research · 27 AugPlanetary prediction engine: Automating global models via Earth AIResearch paper

OpenAI · 27 AugBetter answers, broader thinking: What students gain from ChatGPT and critical-thinking trainingResearch paper

LG AI Research · 27 Aug

[ACL 2026] ReSQL: A Self-Improving Text-to-SQL Framework through Retrieval-Augmented Error Reasoning

Research paper

xAI · 29 AugGrok Bot now works with XCapability change

GitHub · 28 AugGitHub Copilot in Visual Studio — August update