跳到正文
elvis· @omarsar0 · X·· 6 天前AI 评分58
AI 导读

elvis 推荐关注微软等机构关于权重空间编程的研究,并指出更大的模型(Opus 5.5)表现更好、更高效。该研究让 Claude Opus 5.5 直接在代码中写出 Transformer 权重,无需梯度训练;在算法任务上,能解出任务的最小模型中位数仅 166 个参数,26 次运行在留出数据上全部达到 100% 准确率。模型规模影响明显,Sonnet 5.5 中位需 41,600 个参数,Haiku 5.5 需 100,000 个;语言任务更难,参数量停留在约 190,000 且准确率不及学习基线。同一循环用于脑数据时,在视觉皮层预测优于现有可解释模型,并在语言 fMRI 数据中发现一处混淆因素。

正文 · 原文

Recommend reading following up on the idea of weight-space programming. Bigger models (Opus 5.5) tend to do better and be more efficient.

【引用 @dair_ai】Super interesting work from Microsoft and colleagues.

They take a closer look at the exciting research area of weight-space programming.

They had Claude Opus 5.5 write transformer weights directly in code, with no gradient training.

The smallest models that solved the algorithmic tasks had a median of 166 parameters.

The method is called interpretable weight-space programming. An agent proposes a computation, writes it as explicit weights and features, tests it, and revises it for 30 to 50 rounds. A person can read the result line by line.

All 26 runs on algorithmic tasks reached 100% accuracy on held-out data, often by writing a known algorithm into the weights.

They find that the model matters. Sonnet 5.5 needed a median of 41,600 parameters, and Haiku 5.5 needed 100,000.

Language tasks were harder. Those models stayed near 190,000 parameters and lost accuracy against learned baselines.

They then used the same loop on brain data. In the visual cortex, it predicted responses better than existing interpretable models. In language fMRI, it found a confound in the dataset.

https://arxiv.org/abs/2611.18114

Chat with Paper: https://academy.dair.ai/papers/interpretable-weight-space-programming-builds-transparent-transformers-and-brain-2611.18114

引用DAIR.AI@dair_ai
Super interesting work from Microsoft and colleagues. They take a closer look at the exciting research area of weight-space programming. They had Claude Opus 5.5 write transformer weights directly in code, with no gradient training. The smallest models that solved the algorithmic tasks had a median of 166 parameters. The method is called interpretable weight-space programming. An agent proposes a computation, writes it as explicit weights and features, tests it, and revises it for 30 to 50 rounds. A person can read the result line by line. All 26 runs on algorithmic tasks reached 100% accuracy on held-out data, often by writing a known algorithm into the weights. They find that the model matters. Sonnet 5.5 needed a median of 41,600 parameters, and Haiku 5.5 needed 100,000. Language tasks were harder. Those models stayed near 190,000 parameters and lost accuracy against learned baselines. They then used the same loop on brain data. In the visual cortex, it predicted responses better than existing interpretable models. In language fMRI, it found a confound in the dataset. https://arxiv.org/abs/2611.18114 Chat with Paper: https://academy.dair.ai/papers/interpretable-weight-space-programming-builds-transparent-transformers-and-brain-2611.18114
在 X 查看被引用的帖子

来源:elvis · x.com