跳到正文
Lisan al Gaib· @scaling01 · X·· 18 小时前AI 评分18
AI 导读

我发誓,如果我不是 looper、dirty looper 和 loopmaxxer 的发明者

正文 · 原文

I swear if I'm not the inventor of looper, dirty looper and loopmaxxer

【引用 @scaling01】As I said. It's much too late. The loopers are already here.

This one is a much bigger and more well trained looping model.

It has pretty much the architecture I expected GPT-6 to have with the middle half of layers looped, block attention residuals and input gates for the previous loop's message.

(Locally I only tried a small looping model with different positions for the recurrent block. I used "Hyperloop" which also adds some residual stream expressiveness with hyperconnections and gates. Well, but the Inkling one is clearly better than whatever I tried lol)

Fable 5.2's architecture visualization is a bit messy.

引用Lisan al Gaib@scaling01
As I said. It's much too late. The loopers are already here. This one is a much bigger and more well trained looping model. It has pretty much the architecture I expected GPT-6 to have with the middle half of layers looped, block attention residuals and input gates for the previous loop's message. (Locally I only tried a small looping model with different positions for the recurrent block. I used "Hyperloop" which also adds some residual stream expressiveness with hyperconnections and gates. Well, but the Inkling one is clearly better than whatever I tried lol) Fable 5.2's architecture visualization is a bit messy.
在 X 查看被引用的帖子

来源:Lisan al Gaib · x.com