Hacker News AIT2·Open Source
72 pts
A developer released an open-source project that reimplements the DeepSeek V3 architecture from scratch in roughly 190 lines of pure JAX, covering Multi-head Latent Attention, decoupled RoPE, DeepSeekMoE and auxiliary-loss-free load balancing, and pretrains it on a MacBook Air. With no Flax, Equinox or PyTorch dependency and parameters as plain dicts of arrays, the code is readable top to bottom, making it a high-value reference for understanding frontier MoE architectures.