Termsslug: nki-megakernel

NKI megakernel

Fusing multiple layer operators into a single Neuron Kernel Interface kernel to cut decode launch overhead.

Curated Intelligence

1 items · Page 1/1
Hacker News AIT2·Open Source
72 pts
Open-Source TrnRT Brings LLM Inference to AWS Trainium Chips

runanywhere.ai released TrnRT, an open-source inference engine that serves an OpenAI-compatible API from a single trn2.48xlarge instance on AWS Trainium, using NKI layer megakernels for decode, 1M-token sparse/linear attention, and a radix-tree prefix cache spanning device and host memory. GLM-5.3-Flash is the first supported model, with AgentX coding benchmark results published alongside.