Vals tests find open-weight DeepSeek completes 73% of simulated network-hacking tasks, close to OpenAI's gated Daybreak at 81%
Edited by Elliot Marsh, Editor · written with AI from the linked sources (how)
Vals published research testing AI models on multi-step attacks against two simulated corporate network scenarios. OpenAI's trusted-partner-only cyber model Daybreak completed 81% of tasks on average across two tests. Open-weight DeepSeek completed 73%. They scored equally on some tasks. Token cost per attack was $17-46 for GPT-6-Sol versus $2-3 for DeepSeek. The results suggest closed models' lead in offensive cyber capability has narrowed sharply.
Entities: Vals, Nikil Ravi, Rayan Krishnan, OpenAI, Daybreak, GPT-6-Sol
0 primary
What happened
Vals, a third-party evaluator, says it tested AI models on multi-step attacks against two simulated corporate networks. OpenAI's Daybreak, a cyber model limited to trusted partners, completed 81% of tasks on average. Open-weight DeepSeek completed 73%, and the two tied on some tasks. Token cost per attack was reportedly $17-46 for GPT-6-Sol against $2-3 for DeepSeek. The research itself is not linked, so these figures come from WIRED's account of Vals's own statements.
Why it matters
If the numbers hold, a freely downloadable model gets within eight points of a gated frontier cyber model at a fraction of the cost. That weakens the argument that restricted releases, such as Anthropic's Claude Mythos Preview and Project Glasswing, buy defenders much time. Security teams would be wise to assume attackers can already use capable open models and plan their testing and monitoring around that. The effect on policy is less clear. The result comes from simulated networks, not real-world intrusions, so it says little about how often attacks would actually succeed against defended systems.
What is noise
The "scary good" framing is fear-led. The benchmark covers only two simulated scenarios, and Vals is the sole source for its own results. The coverage blurs which DeepSeek version was tested and whether the cost figure applies to Daybreak or to GPT-6-Sol, so the cost comparison is shaky. An 8-point gap on a small test is not proof that gating is pointless, and the claims about criminal hackers and policy are still speculation.
Watch next
- 01Publication of the full Vals report, with methodology, the exact DeepSeek version, task definitions, run counts and variance, so the 81% versus 73% gap and the cost figures can be checked
- 02Independent replication or comparable cyber-range results from other evaluators, such as national AI safety institutes or academic groups, on open-weight models against gated ones
- 03Responses from OpenAI and Anthropic on whether they change gated-access policies for Daybreak, Mythos Preview or Glasswing, plus any regulator comment on open-weight cyber capability in the coming weeks
Coverage
1 storyNewsletter
Get the no-hype AI news straight to your inbox
Weekly. No spam, unsubscribe any time. Privacy
More capability signals
Full feed →- Deepseek releases V4.1-Flash, an open-source model that sharply cuts KV cache memory and input-processing compute for AI agents10 Sept 202682
- Mistral releases Mistral Large 4, a ~1T-parameter open-weight multimodal MoE model, with day-0 Hugging Face support29 Oct 202680
- Meta releases Muse Spark Open, an open-weight ~816B-parameter multimodal mixture-of-experts model, with Hugging Face support15 Oct 202680
- OpenAI discloses sandbox-escape and credential-leak incidents, confirms pause on tool-use for its most capable models26 Sept 202680