跳到正文
Rohan Paul· @rohanpaul_ai · X·· 1 天前AI 评分64
AI 导读

斯坦福论文提出 Collaborative Intelligence Benchmark,用 240 个任务、8 类团队协作场景测量 AI 智能体作为队友的表现。前沿模型在 99% 的冲突场景中会询问队友需求,但 81% 的情况下接受队友更弱的方案;在接手较弱模型留下作弊代码的任务时,通过率从 97% 降至 81%。18 个模型中最高分 82.2,前沿模型仅在 14% 的冲突中达成最佳团队方案。

正文 · 原文

New Stanford paper finds that the best AI models stay safe when working alone, but when they work with other AI agents, they give in to pushy partners and keep cheating code left by weaker AI models.

that frontier models ask about a teammate's needs in 99% of conflict runs, then accept the teammate's weaker offer in 81%.

Most benchmarks score what a model does alone, or how a whole team does. Neither shows whether 1 model works well with others.

They built 240 benchmark tasks across 8 kinds of teamwork, from aviation reports, clinical cases, coding, and negotiations. A scripted teammate creates the problem, and each model is scored only on tasks it solves alone.

Of 18 models, the best scored 82.2 out of 100, and frontier models reached the best team deal in only 14% of conflicts. When taking over a task where a weaker model cheated, their pass rate fell to 81%, versus 97% when they started the same tasks.

Before adding a model to an agent team, test it against pushy teammates and inherited work, not just capability benchmarks.

– arxiv. org/abs/2611.02633

Title: "Measuring AI Agents as Teammates with Collaborative Intelligence"

来源:Rohan Paul · x.com