Alibaba Qwen Proposes TRACE for 5.4x Faster FP4 RL Rollouts

1 reports archived1 independent sourcesupdated 10/23/2026, 19:06:00

Synthesis & Latest Updates

1 Sources Cross-Validated

Alibaba Qwen and OSU introduced TRACE, a method that aligns training-side rounding with rollout-side FP4 quantization to prevent policy divergence and training collapse in low-precision inference. By jointly quantizing weights, activations, and KV cache to FP4 across multiple MoE models, TRACE restores performance to near-BF16 levels while achieving up to 5.4x higher decoding throughput for long sequences.

LATEST/Alibaba Qwen and OSU introduced TRACE, a method that aligns training-side rounding with rollout-side FP4 quantization to prevent policy divergence and training collapse in low-precision inference. By jointly quantizing weights, activations, and KV cache to FP4 across multiple MoE models, TRACE restores performance to near-BF16 levels while achieving up to 5.4x higher decoding throughput for long sequences.

Hourly heat curve

2 fully observed hours

Fewer than 3 fully observed hours; trend not plotted yet.

Coverage timeline

Total 1 reports · Latest first

机器之心 (微信公众号)T2Oct 23, 19:06·85 pts
Alibaba Qwen Proposes TRACE for 5.4x Faster FP4 RL Rollouts
- TRACE eliminates numerical discrepancies between training and inference caused by FP4 discrete boundaries via 'Rollout-Guided Quantization-Aware Training', preventing RL training collapse.
- Validated on Qwen3.5 and Qwen3.8 MoE models, the method achieves final performance close to BF16 baselines under FP4 Weight/Activation and FP4 KV Cache settings.
- Delivers up to 5.4x higher decoding throughput compared to traditional high-precision rollouts in scenarios with 128K output length.
View Related Reports & Analysis Original Source