Tencent Xuanwu Lab's Atyun AI achieved 84.0% on the CyberGym cybersecurity benchmark, surpassing Anthropic's Claude Mythos Preview, while consuming less than 0.1% of Mythos's budget. This demonstrates that a cost-effective, open-source-based AI can outperform a major proprietary model in vulnerability discovery, potentially lowering barriers for organizations to adopt AI-driven security auditing. Atyun AI is built on the locally deployable open-source model GLM-5.1 and discovered multiple high-severity logical vulnerabilities in projects like curl, gnark, OpenSSL, Python cryptography, and Java bc-java, with scores up to 9.3. It ranked 1st in critical vulnerability severity on the Berkeley BVI real-world vulnerability leaderboard.
Background
CyberGym is a large-scale benchmark introduced by UC Berkeley that includes 1,507 real-world vulnerabilities across 188 open-source software projects to evaluate AI agents' cybersecurity capabilities. GLM-5.1 is a flagship open-source LLM by Z.AI, designed for long-horizon autonomous tasks. Claude Mythos Preview is Anthropic's specialized cybersecurity model under the "Glass Wing Project."