Thinking Machines Lab 开源了 594B 参数的 Inkling 2 Small，该模型从预训练第一步到最终 RL rollout 全程使用原生 4-bit 前向计算，已在 HuggingFace 以 Apache 2.0 许可开放下载。
材料给出了 4-bit 端到端训练的提速幅度、精度代价和显存门槛，可据此判断低比特训练的实际取舍。
Thinking Machines Lab kept its promise and open-sourced Inkling 2 Small, a 594B-parameter model trained with 4-bit forward passes.
> now available for download on HuggingFace under an Apache 2.0 licens
> Training in 4-bit cut pretraining time 8% and RL time 20%, for about 2% efficiency loss.
Running it takes at least 500GB of GPU memory in NVFP4 and at least 800GB in MXFP8, and MXFP8 gains “a very marginal 1–2 percentage points” over NVFP4 on evals.
Because training kept the same numerics from pretraining through the final RL rollout, the released 4-bit checkpoint needed no separate calibration step.
The lab first tried NVFP4 in both forward and backward passes, but that hurt compute efficiency too much, so only the forward pass stays in 4-bit.
Training with fewer bits is faster, but quantization error can interfere with learning. Today on Connectionism, we detail how we trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout. https://thinkingmachines.ai/blog/end-to-end-4-bit-training在 X 查看被引用的帖子
来源：Rohan Paul · x.com