跳到正文
Rohan Paul· @rohanpaul_ai · X·· 8 天前AI 评分71
AI 导读

卡内基梅隆大学等机构的论文提出 Singularity-Bench,用预训练、微调、推理、智能体 harness、集群编排等 6 项端到端任务评测智能体自建 AI 系统的能力,并在相同 GPU 上与 SGLang、Claude Code 等生产系统对比。

正文 · 原文

New Carnegie Mellon paper finds that Opus 5.5 + Claude Code built a pre-training pipeline, an inference engine, and a cluster orchestrator that matched or beat production tools like SGLang and NVIDIA Dynamo on the same GPUs.

Singularity-Bench gives agents 6 end-to-end jobs, from pre-training a model to building a serving engine. Each result is scored against a production system, such as SGLang or Claude Code, on the same GPUs.

Opus 5.5 scored a Singularity-Index of 1.25, where 1 equals the human baselines, while the other 10 agents averaged 0.58. Its serving engine earned 11.52 USD per GPU-hour against 8.18 for SGLang.

来源:Rohan Paul · x.com