Alex Zhang 团队论文指出,LLM Harnesses 作为组合泛化器,能通过组合现有能力使模型在任务长度、策略及递归深度上实现更强泛化。该机制通过保持轨迹局部分布内(LID),利用合成数据与代理训练替代长流程 rollout,显著提升长任务的数据效率。
Highly recommended read. Learn to build your own harness, folks.
My belief—and that of many fully AI-native builders and companies—is that harnesses remain largely underexplored.
Research shows a huge edge in custom harnesses.
But little is known about why.
Alex and colleagues have been exploring this direction and have noted that harnesses can act as compositional generalizers.
In other words, harnesses can push LLMs further (with stronger generalization effects) through existing LLM capabilities.
Harnesses like RLMs are a good recent example, with abstractions that unlock this.
I have recently started seeing success in post-training used to co-evolve harnesses and models. I think this could explain why this is happening, but more research is needed to understand this better.
I have experienced this with a variant of dynamic workflows I have implemented in my meta harness.
Harnesses used to be a crutch for weak LLMs. In 2026 and onward, with stronger models, harnesses now offer a mechanism for composing an LLM’s existing and new capabilities to generalize well beyond their training environments. In our new paper, LM Harnesses are Compositional Generalizers, we explore how various harness abstractions enable LM systems to generalize over task length, task strategies, recursive task depth, and more through composing existing LLM capabilities. These abstractions provide a means for more data-efficient training, especially over long tasks, by keeping trajectories locally in-distribution (LID) — i.e., shaping all individual LLM calls to be in-distribution. For tasks that take a very long time to roll out, this implies that we can even perform surrogate harness training, meaning we train an individual LLM call instead of doing the very long harness rollout, using purely synthetic tasks, with fake tool calls + sub-agents! A common idea floating around is that LLMs will absorb harnesses and make them obsolete. We disagree, and argue that we haven’t sufficiently explored what harnesses can provide. The full paper, logs, codebase, and data are available: https://www.alphaxiv.org/abs/2611.harness-cg w/ @SOURADIPCHAKR18 @lateinteraction在 X 查看被引用的帖子
来源:elvis · x.com