From the source
Today we're open sourcing supabase/evals , our benchmark and framework for testing how well AI agents build using Supabase.
It runs coding agents including Claude Code, Codex, and OpenCode against real Supabase tasks, for example, building a schema, debugging a failed Edge Function, or fixing a broken RLS policy, and then scores how well they performed.
It powers both our published benchmark and an internal regression suite we monitor daily.
As more people ship Supabase projects through an agent instead of by hand, we wanted a way to measure that experience instead of guessing.
The results cited here reflect a snapshot at the time of writing.
AI tooling changes rapidly, so check the live page for current results.
Why we built this # Agents are becoming a primary way people build with Supabase, interacting through our CLI, MCP server, agent skills, and docs.
We needed a way to understand how agents perform across all of these Supabase surfaces, not just one tool in isolation.
By tracking where agents stumble, we can make intentional fixes and verify they hold instead of regressing, and test new features confidently before shipping.
How we approached it # …
