JAX

A developer released an open-source project that reimplements the DeepSeek V3 architecture from scratch in roughly 190 lines of pure JAX, covering Multi-head Latent Attention, decoupled RoPE, DeepSeekMoE and auxiliary-loss-free load balancing, and pretrains it on a MacBook Air. With no Flax, Equinox or PyTorch dependency and parameters as plain dicts of arrays, the code is readable top to bottom, making it a high-value reference for understanding frontier MoE architectures.