哦这太牛了 但考虑到它的效率低了好几倍,我不太会说它击败了 Sol 我想说的是,它击败了 Kimi-K3、GLM-5.3 Flash 和 Grok-4.7,这已经令人印象深刻了
We found DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on our multi-step offensive cybersecurity task. Sol was tested via the Daybreak trusted access program. We have been building a cybersecurity benchmark based on simulated corporate networks. This is a valuable indicator for the cyber-risk a model poses if in the hands of a bad actor. We plan to make the full evaluation public in the coming weeks but wanted to share our first immediate finding here.