From the source
Reflection AI says it partnered with Gray Swan to red-team its open model Beam, testing safeguards against adversarial attacks and prompt injection in agentic systems.
- The company describes two distinct workflows for responsible open-model development: finding harmful behaviors before deployment, and checking that a model stays secure once it runs as an agent.
- Gray Swan's automated system Shade put Beam under sustained adversarial pressure across cyber, biology, chemistry, mass persuasion and influence operations, and general harmful content, making up to 200 attempts per scenario.
- In the cyber domain, Gray Swan tested 100 text-only scenarios mapped to MITRE ATT&CK techniques, including 54 malicious and 46 dual-use cases, and separately checked whether Beam gave functionally effective help on offensive cyber tasks as a coding agent.
- Reflection says the results showed where Beam's safeguards held and where more work is needed, and warns that low per-attempt success can still leave working paths for persistent attackers.
- The company also co-sponsored Gray Swan's latest public IPI Black arena challenge, and ran its own internal prompt-injection evaluation documented in its Safety Technical Report.



