From the source
Apple researchers present Agent Seer, a pipeline that synthesizes realistic evaluation scenarios for tool-using AI agents from a single Model Context Protocol (MCP) specification without examples, live tool access, or domain-specific tuning.
- The pipeline generates graded scenarios and multi-turn dialogues, achieving strong tool-calling correctness and conversational coherence across seven MCP specifications.
- The analysis finds that parameter schema complexity is the strongest correlate of quality variation, with argument value accuracy as the dominant failure mode.