AWS Machine Learning BlogT1·Infra
62 pts
Exa optimized its embedding infrastructure on AWS Inferentia2 and Trainium2 using the Neuron SDK and custom NKI kernels, achieving a 50% throughput gain for indexing tens of billions of documents. The post stresses that per-document embedding cost dominates unit economics at web scale, and that pooling heterogeneous hardware with low-level kernel tuning yields durable savings.