Agile Deliberation: How AI helps humans define the gray areas

Google Research introduced Agile Deliberation, a framework that treats building subjective AI image classifiers as an iterative conversation in which the system helps users sharpen their own definitions before training.

• The work addresses a gap in standard workflows, which assume a labeler already holds a clear, stable definition of a category, an assumption that fails for concepts like healthy food or inspiring artwork whose boundaries shift with culture and policy.
• Agile Deliberation splits the task into concept scoping, where a broad idea is broken into sub-categories and explicit exclusions, and concept iteration, where vision-language models surface semantically borderline images that challenge the current text definitions.
• Human feedback on those edge cases feeds an automatic prompt optimization loop, so the classifier's boundaries move with the user's refined intent.
• Google Research says it bypassed simulated testing because static benchmarks cannot capture subjective concepts, and instead ran 18 live 1.5-hour sessions with human participants.
• The team reports the approach reduced user fatigue and anxiety by replacing manual prompt engineering with direct feedback on borderline images, producing more consistent models that outperform standard baselines.
• The underlying code and algorithms were open-sourced on GitHub.