Thinking Machines Open-Sources 594B-Parameter 4-Bit Trained Model
Thinking Machines Lab has fulfilled its promise by open-sourcing Inkling 2 Small under Apache 2.0, a 594B-parameter model trained with native 4-bit forward passes from the first pretraining step through the final RL rollout. The 4-bit approach cut pretraining time by 8% and RL time by 20% for roughly 2% efficiency loss, and the released checkpoint requires no separate calibration.
Full Report
Rohan Paul (@rohanpaul_ai) · 2026-11-13 23
Thinking Machines Lab kept its promise and open-sourced Inkling 2 Small, a 594B-parameter model trained with 4-bit forward passes.
now available for download on HuggingFace under an Apache 2.0 licens Training in 4-bit cut pretraining time 8% and RL time 20%, for about 2% efficiency loss.
Running it takes at least 500GB of GPU memory in NVFP4 and at least 800GB in MXFP8, and MXFP8 gains “a very marginal 1–2 percentage points” over NVFP4 on evals.
Because training kept the same numerics from pretraining through the final RL rollout, the released 4-bit checkpoint needed no separate calibration step.
The lab first tried NVFP4 in both forward and backward passes, but that hurt compute efficiency too much, so only the forward pass stays in 4-bit.
↳ 引用 @thinkymachines (Thinking Machines): Training with fewer bits is faster, but quantization error can interfere with learning.
Today on Connectionism, we detail how we trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout. https://thinkingmachines.ai/blog/end-to-end-4-bit-training
