跳到正文
elvis· @omarsar0 · X·· 1 天前AI 评分58
AI 导读

zavoai 发布 Hospital-bench 智能体基准,用于测试 AI 智能体在模拟医疗系统中的医院运营工作,覆盖 13 个应用和 131 个工具。20 个模型完成 48 项任务,GPT-6.2 Astra 以 62.4% 的平均分领先,Claude Opus 5.5 完全通过的任务最多(17/48)。

正文 · 原文

Introducing Hospital-bench, a new agent benchmark from @zavoai.

It tests hospital operations work. Agents handle coverage checks, prior authorizations, claims, and records requests inside a simulated health system with 13 apps and 131 tools.

20 models ran 48 tasks. GPT-6.2 Astra led with a 62.4% mean score, while Claude Opus 5.5 fully passed the most tasks (17/48).

Safety was a major challenge. Models made errors, including disclosing restricted patient records, incorrectly modifying paid claims, and acting beyond their authority.

GPT-6.2 Astra and Gemini 4 Neon had no tasks zeroed due to harm, while some models had as many as 10.

The results show that completing complex, multi-system workflows reliably and safely remains a major challenge for AI agents.

引用Can Zehebi@canz101
Introducing Hospital-bench: the most comprehensive hospital operations simulation built to evaluate AI agents. Behind every hospital visit are hours of administrative work: insurance verification, prior authorizations, claims, billing, patient records, and paperwork spread across dozens of workflows. We built Hospital-bench to test whether AI agents can handle this complexity in a realistic hospital environment. Here are the results ↓
在 X 查看被引用的帖子

来源:elvis · x.com