From the source
CyberSecEval 2, a comprehensive evaluation framework for cybersecurity risks and capabilities of Large Language Models (LLMs).
- It includes benchmarks for insecure code generation, prompt injection, compliance with cyber attack requests, code interpreter abuse, and automated offensive cybersecurity capabilities.
- The post also provides key insights from evaluating state-of-the-art LLMs, noting a decrease in compliance with cyber attack requests since the first version, but highlighting ongoing challenges with prompt injection and interpreter abuse.
