Moore Threads announced Day-0 adaptation of Moonshot's open-weight Kimi K3.1 Pro on its MTT S5000 AI card and MUSA software stack, covering model loading, service startup, and text/image inference. Key work includes adapting sparse attention DSA indexing and KV cache layout, online FP8 conversion of MXFP4 expert weights, and integrating visual attention. It signals faster domestic GPU software-stack readiness for trillion-parameter MoE models.
Termsslug: low-precision-quantization
Low-Precision Quantization
低精度量化
Representing weights and activations with fewer bits to reduce memory footprint and bandwidth demand while preserving accuracy.
Curated Intelligence
IT之家 · 智能时代T2·Infra
58 pts
IT之家 · 智能时代T2·Infra
78 pts
Nvidia published a technical explainer arguing that a full-stack co-design approach can ease the HBM crunch and memory wall. Its stack combines NVFP4 low-precision compression, weight sharding across up to 72 GPUs in NVL72, LPDDR buffering on Vera CPUs, Dynamo context migration and CMX context storage to move inactive context off HBM. Citing the SemiAnalysis AgentX benchmark, Nvidia claims Vera Rubin NVL72 delivers up to 30x agent throughput per dollar versus GB300 NVL72 while HBM capacity per GPU grows only about 3%, signaling a shift from raw memory capacity to effective tokens per byte and per watt.