我发誓,如果我不是 looper、dirty looper 和 loopmaxxer 的发明者
I swear if I'm not the inventor of looper, dirty looper and loopmaxxer
【引用 @scaling01】As I said. It's much too late. The loopers are already here.
This one is a much bigger and more well trained looping model.
It has pretty much the architecture I expected GPT-6 to have with the middle half of layers looped, block attention residuals and input gates for the previous loop's message.
(Locally I only tried a small looping model with different positions for the recurrent block. I used "Hyperloop" which also adds some residual stream expressiveness with hyperconnections and gates. Well, but the Inkling one is clearly better than whatever I tried lol)
Fable 5.2's architecture visualization is a bit messy.
As I said. It's much too late. The loopers are already here. This one is a much bigger and more well trained looping model. It has pretty much the architecture I expected GPT-6 to have with the middle half of layers looped, block attention residuals and input gates for the previous loop's message. (Locally I only tried a small looping model with different positions for the recurrent block. I used "Hyperloop" which also adds some residual stream expressiveness with hyperconnections and gates. Well, but the Inkling one is clearly better than whatever I tried lol) Fable 5.2's architecture visualization is a bit messy.在 X 查看被引用的帖子
来源:Lisan al Gaib · x.com