Hacker News AIT2·Open Source
72 pts
runanywhere.ai released TrnRT, an open-source inference engine that serves an OpenAI-compatible API from a single trn2.48xlarge instance on AWS Trainium, using NKI layer megakernels for decode, 1M-token sparse/linear attention, and a radix-tree prefix cache spanning device and host memory. GLM-5.3-Flash is the first supported model, with AgentX coding benchmark results published alongside.