Greg Brockman: 引用推文 @gdb(仅供理解上下文): 我帮助创立 OpenAI,是因为我希望 AGI 对人类安全且有益。这篇文章提供了透明度,可以作为行业共享安全基线的基石。 文章中最重要的概念或许是以下这句:“我认为 AI 也将显著促进对齐和监控方面的进展,但这种进展不太可能足够快,以允许预训练潜力的全面加速。” 为了确保安全能跟上预训练模型通用智能的进步(以“困惑度”衡量),文章提出了“基于困惑度和其他测量的共同限制,开发者承诺在我们的理解和监控 AI 行为的能力达到某个具体里程碑(例如开发出 CoT 监控器的继任者)之前,不跨越这些限制。” 总体思路是创建第一个共享限制,针对某些共享的关键指标,并附带具体的退出标准,从而允许超越该限制。这可以导致“设定连续限制并为进一步发展定节奏的共同框架”。 很高兴推进关于迈向根植于科学的共享标准的节奏的讨论!
Greg Brockman:
【引用 @gdb】I helped start OpenAI because I want AGI to be safe and beneficial for people. This post provides transparency that can be the foundation of industry-wide shared safety bars.
Perhaps the most important concept from the post is the following: "AI will also significantly contribute to progress on alignment and monitoring, I think that progress is unlikely to be fast enough to allow the full potential acceleration in pretraining."
Towards ensuring that safety keeps pace with general intelligence of pretrained models (as measured by "perplexity"), the post proposes "common limits, based on perplexity and other measurements, that developers are committed to not crossing until our ability to understand and monitor AI behavior achieves a certain concrete milestone (such as the development of a successor to CoT monitors)."
The general idea is to create a first shared limit, on some shared key metric, along with concrete exit criteria that would allow going beyond that. This can lead to a "common framework for setting successive limits and pacing further development".
Excited to move forward the conversation on pacing towards shared standards that are rooted in the science!
I helped start OpenAI because I want AGI to be safe and beneficial for people. This post provides transparency that can be the foundation of industry-wide shared safety bars. Perhaps the most important concept from the post is the following: "AI will also significantly contribute to progress on alignment and monitoring, I think that progress is unlikely to be fast enough to allow the full potential acceleration in pretraining." Towards ensuring that safety keeps pace with general intelligence of pretrained models (as measured by "perplexity"), the post proposes "common limits, based on perplexity and other measurements, that developers are committed to not crossing until our ability to understand and monitor AI behavior achieves a certain concrete milestone (such as the development of a successor to CoT monitors)." The general idea is to create a first shared limit, on some shared key metric, along with concrete exit criteria that would allow going beyond that. This can lead to a "common framework for setting successive limits and pacing further development". Excited to move forward the conversation on pacing towards shared standards that are rooted in the science!在 X 查看被引用的帖子
来源:Andrew Curran · x.com