MosaicLeaks: Can your research agent keep a secret?

Introduces MosaicLeaks, a benchmark for evaluating privacy leakage in deep research agents, and proposes Privacy-Aware Deep Research (PA-DR) training that reduces answer/full-information leakage from 34.0% to 9.9% while improving strict chain success from 48.7% to 58.7%.

From the source
MosaicLeaks proposes a new deep-research task with multi-hop questions that interleave public and private information. Across the models we tested, agents frequently leaked private information, and training only for task performance made it worse. We propose a mosaic-leakage-aware RL training method, Privacy-Aware Deep Research (PA-DR), which raises strict chain success (the share of chains where every hop is answered correctly) from 48.7% to 58.7% while reducing answer/full-information leakage from 34.0% to 9.9%.
huggingface.co

Visit source ↗huggingface.co