Hacker News AIT2·78 pts
- A single Cerebras WSE-3T chip has 44GB of on-chip memory, insufficient for trillion-parameter model weights which require terabytes.
- By sharding models across multiple chips, Kimi K3 (3T parameters) achieves inference speeds of 1000-2000+ tokens/s.
- The post explains strategies for fitting models into on-chip memory, mapping across chips, and handling high concurrent requests.