Termsslug: expert-streaming

Expert Streaming

专家流式加载

Inference technique that loads MoE expert weights from SSD on demand to bypass memory limits.

Curated Intelligence

1 items · Page 1/1
Hugging Face BlogT1·Tools
78 pts
180B-Parameter MoE Model Runs on 32GB of RAM via SSD Expert Streaming

Hugging Face blogger Tarek Ziadé demonstrates running a 180B-parameter MoE model on a 32GB unified-memory MacBook M5 by streaming expert weights directly from SSD instead of keeping them resident in RAM. The approach lands within roughly 6% of the dedicated Strata expert-streaming solution on token generation speed, sharply lowering the hardware bar for running very large MoE models on consumer devices.