Companiesslug: runanywhere-ai

runanywhere.ai

Team focused on cross-hardware LLM inference deployment and developer of TrnRT.

Curated Intelligence

1 items · Page 1/1
Hacker News AIT2·Open Source
72 pts
Open-Source TrnRT Brings LLM Inference to AWS Trainium Chips

runanywhere.ai released TrnRT, an open-source inference engine that serves an OpenAI-compatible API from a single trn2.48xlarge instance on AWS Trainium, using NKI layer megakernels for decode, 1M-token sparse/linear attention, and a radix-tree prefix cache spanning device and host memory. GLM-5.3-Flash is the first supported model, with AgentX coding benchmark results published alongside.

runanywhere.ai | Today for AI