跳到正文
elvis· @omarsar0 · X·· 1 天前AI 评分58
AI 导读

AItonomy Foundation 发布 Science's Last Exam 基准,用来自 9 个科学领域的真实研究任务考察智能体能否提出假设、规划实验流程、运行科学代码与软件并解读实验视频。任务由 30 多家机构的研究者编写和评审,初始榜单上最强配置(Claude Code 中的 Claude Opus 5)综合得分 37.5/100,端到端研究赛道所有模型均为 0 分。

正文 · 原文

Can AI agents do scientific research on their own?

Science's Last Exam is a new benchmark measure this. It's built from real research tasks across nine scientific fields.

It tests whether an agent can form hypotheses, plan protocols, run scientific code and software, and read what happens in experiment videos. Researchers from more than 30 institutions wrote and reviewed the tasks.

On the initial leaderboard, the strongest agent configuration (Claude Opus 5 in Claude Code) reaches a composite score of 37.5 out of 100. On the end-to-end research track, every model scores 0.

引用AItonomy Foundation@aitonomyfound
I’m incredibly proud to share what we’ve been building at the AItonomy Foundation: Science’s Last Exam (SLE). 🧪✨ Can AI do science? Instead of Q&A, we give agents actual research goals and real-world tools (lab data, software), then grade them against hidden expert checks. Deeply grateful to the brilliant minds at 34 institutions (Harvard, MIT, Stanford, etc.) who helped craft 113 tasks across 9 disciplines. ❤️ The results? Science is hard. Of 8 frontier models, the best scored 37.5/100. On true end-to-end research? All scored 0. We have an exciting journey ahead! 🚀 (Claude Opus 5.5 & GPT-6 Astra coming soon—just clearing bio safeguards). Everything opens Nov 6. 🗓️ 🌐 https://sciences-last-exam.org 💻 https://github.com/Gen-Verse/Science-Last-Exam 🤗 https://huggingface.co/datasets/Gen-Verse/Science-Last-Exam Thread below on how the exam works and where models break. 👇 (1/14) #AGI #AIforScience #AIAgents #SciencesLastExam #OpenAI #Anthropic
在 X 查看被引用的帖子

来源:elvis · x.com