Together AI 发布 Together Compiler 让智能体持续热重载推理前向
Together AI 推出 Together Compiler，让 AI 智能体在生产流量变化时持续重调推理前向计算。其核心是常驻服务引擎配合热重载，无需重启即可加载智能体改写的前向实现，并用 12 层快速筛选加 60 层慢速验证的双层机制确认真实流量收益。官方称在 SemiAnalysis InferenceX 上最多提升 2.6 倍 TPS，但部分场景增益有限且数据均为自报，系统闭源并已上线其无服务器端点。
报道正文
▲ Marktechpost AI · 2026-11-05 08
▲ Together AI just released Together Compiler, a system where AI agents keep re-tuning the inference forward pass as production traffic changes.
The key idea is a persistent engine. The serving engine starts once, and each agent-written forward pass is hot-reloaded with no restart. A 12-layer fast tier screens candidates, and a 60-layer slow tier confirms gains on real traffic. That gives agents 252x more experiments than restarting the engine each time.
On SemiAnalysis InferenceX, it beat the best open configurations by up to 2.6x TPS in under 24 hours per model. At batch size 1, MiniMax M3.1 Flash on B200 hit 1,111 P90 TPS per user vs 633. It is a trade-off though: some gains are small. GLM 5.3 on MI355X moved only from 463 to 480 TPS, and every number is self-reported.
It already runs on Together's serverless endpoints, with a limited beta for dedicated endpoints. Not open source. Tested on Kimi K3, GLM 5.3, MiniMax M3.1 Flash and DeepSeek v4.1 Flash across B200, B300, GB300, MI355X and Trainium2......
Full analysis:
