Toolsslug: strata

Strata

MoE expert-streaming inference solution for consumer hardware, used as the performance baseline here.

Curated Intelligence

1 items · Page 1/1
Hugging Face BlogT1·Tools
78 pts
180B-Parameter MoE Model Runs on 32GB of RAM via SSD Expert Streaming

Hugging Face blogger Tarek Ziadé demonstrates running a 180B-parameter MoE model on a 32GB unified-memory MacBook M5 by streaming expert weights directly from SSD instead of keeping them resident in RAM. The approach lands within roughly 6% of the dedicated Strata expert-streaming solution on token generation speed, sharply lowering the hardware bar for running very large MoE models on consumer devices.