@ylecun 团队提出 Slow SVReg 解决 JEPA 坍塌问题,成功让模型在潜在空间构建世界地图。该方法通过慢通道和门控残差预测器平衡内容与几何约束,在 1P Medium maze 取得 92.9% 成功率,城市行走定位误差仅 1.6 米。
Can a JEPA learn a map of the world in its "head"?
This new paper by @ylecun, @mengyer, @yingwww_, @choang333, and @KuangYilun says yes, but only if you change how JEPA avoids collapse.
What's interesting – the researchers firstly proposed a fix, called SVReg, but then discovered its content–geometry tradeoff and upgraded the solution.
Here are the details:
A planner in latent space needs two things:
- Paths have to be smooth, so distance tells you if you're getting closer. That's geometry.
- And the cube you're trying to move has to still be on the map. That's content.
Typical stop-gradient and SIGReg often give you one and lose the other.
SVReg stops the model from giving every frame the same code. It checks the spread along random directions and only steps in when it gets too small. Without stop-gradient, the model can also change what it encodes, dropping details it can’t reliably predict.
But the title itself says that there is "content–geometry tradeoff".
The same predictability pressure erases things that change slowly and are hard to predict, like a block that sits still until the agent pushes it.
So they built Slow SVReg, with 2 additions:
- Slow channels: A few fixed dimensions get penalized if they change too fast between frames (an idea borrowed from slow feature analysis).
- Gated residual predictor: Predicts the next state as current + gate × change (with the gate starting at 0.1), so unchanging features cost nothing to predict.
The results:
- 1P Medium maze: 92.9% success (vs. 21.2% stop-gradient) with way smoother paths (1.20 vs. 5.31 jaggedness).
- PushT: 87.3% success (vs. 44.0% stop-gradient).
- Efficiency: Nearly maxes out hard U-maze goals using just 20% of the data.
And the most interesting part is city walks.
Trained only on first-person frames and actions from 3D city blocks (no pose data), Slow SVReg pinpoints the agent within 1.6 meters (beating stop-gradient's 3.4 m and an untrained encoder's 25.9 m). The street layout pops right out in the plain principal components.
So the JEPA line keeps circling the same question – what makes a latent good enough to plan in? LeJEPA made anti-collapse provable, LeHami added physics as a constraint. This paper shows the anti-collapse choice itself decides whether you get a map or a cloud of points.
来源:Turing Post (Ksenia Se) · x.com