跳到正文
Lisan al Gaib· @scaling01 · X·· 4 天前AI 评分58
AI 导读

作者分析 OpenAI 近期关于 CoT 可监控性的博客,指出其中透露了一个至少自 2026 年 9 月起就存在的内部模型,强于 GPT-6 Astra。

正文 · 原文

In a recent blog post about CoT monitorability OpenAI revealed some info about an internal model stronger than GPT-6 Astra that has existed at least since September 2026.

The jump from GPT-6 Astra to this internal model in terms of perplexity (often associated with pre-training quality) is about as big as from GPT-5.4 to GPT-5.5.

Despite their plots being mostly unlabeled you can match the points to existing models and extract several evaluation scores.

With this data you could have predicted continued rapid progress on reasoning, especially no-CoT reasoning, before GPT-6.2-Astra was released.

The fit on all models predicts GPT-6.2-Astra's 171 NCRI score exactly. However, a steeper, more speculative fit puts the internal model even higher at 182 and other scores are slightly higher than the ones of GPT-6.2-Astra.

I'm still uncertain whether this actually is 6.2 Astra or another larger pre-train.

If it is 6.2 Astra and has the same size as 6 Astra I think it's more plausible that it is more loopy than 6 Astra. Or they found a more effective training method/arch for loopers.

If it's a larger pre-train I would naively guess that it's a similar sized jump as from GPT-5.4 to GPT-5.5, which based on my estimates was a ~2x in params.

引用Lisan al Gaib@scaling01
NCRI / serial reasoning depth for Astra 6.2 is a higher than I expected so if it's the same underlying model as Astra 6 then it might be more loopy but my default assumption is that they are both based on the same model and loop back two times why? another pass over the recurrent block would increase cost, but prices fell from Astra 6 to 6.2. And the 3 and 4 loop config for Astra 6.2 would execute > 240 layers, which would contradict Jakub's statement. now, what options are still open? - Astra 6 was undertrained and didn't yet utilize the loopies effectively (<-- my favorite) - Astra 6.2 is a larger model - I'm regarded and all of this is wrong
在 X 查看被引用的帖子

来源:Lisan al Gaib · x.com