Toolsslug: aws-neuron-sdk

AWS Neuron SDK

Compiler and inference optimization SDK for Inferentia and Trainium chips, supporting custom NKI kernels.

Curated Intelligence

1 items · Page 1/1
AWS Machine Learning BlogT1·Infra
62 pts
Exa Boosts Embedding Throughput 50% on AWS AI Chips

Exa optimized its embedding infrastructure on AWS Inferentia2 and Trainium2 using the Neuron SDK and custom NKI kernels, achieving a 50% throughput gain for indexing tens of billions of documents. The post stresses that per-document embedding cost dominates unit economics at web scale, and that pooling heterogeneous hardware with low-level kernel tuning yields durable savings.