Tsinghua Unveils BaGuaLu-2: Runtime Adaptation Solves Memory Spikes in Large-Scale Training

1 reports archived1 independent sourcesupdated 10/28/2026, 12:03:00

Synthesis & Latest Updates

1 Sources Cross-Validated

Tsinghua University researchers introduced 'BaGuaLu-2' at SC26, a runtime-adaptive system that dynamically adjusts recomputation and offloading strategies to mitigate memory spikes in MoE training. Without requiring static configuration changes, the system achieved a 1.14x throughput increase and up to 35% MFU on 10,000+ GPU clusters, effectively preventing OOM-induced task failures.

LATEST/Tsinghua University researchers introduced 'BaGuaLu-2' at SC26, a runtime-adaptive system that dynamically adjusts recomputation and offloading strategies to mitigate memory spikes in MoE training. Without requiring static configuration changes, the system achieved a 1.14x throughput increase and up to 35% MFU on 10,000+ GPU clusters, effectively preventing OOM-induced task failures.

Hourly heat curve

4 fully observed hours

Now

8.5

Peak

9.110-29

24h change

–

The line compares only sources observed throughout; gap hours are interpolated to keep the trend continuous.

Coverage timeline

Total 1 reports · Latest first

机器之心 (微信公众号)T2Oct 28, 12:03·78 pts
Tsinghua Unveils BaGuaLu-2: Runtime Adaptation Solves Memory Spikes in Large-Scale Training
- Utilizes a runtime-adaptive mechanism to decide between saving, recomputing, or offloading after gating but before expert execution.
- Eliminates the need for static configurations by automatically handling transient memory spikes caused by MoE load imbalance.
- Achieved an average MFU of 32.9% and total throughput of 2.012 EFLOPS on a cluster of 12,288 Haiguang DCUs.
View Related Reports & Analysis Original Source