NVIDIA has open sourced NeMo Switchyard, a library that dynamically picks which model handles each coding-agent request on Amazon Bedrock, with LiteLLM integration offering strategies such as stage routing. The post argues routing should be judged by cost per solved task rather than per-token price, since agent tasks span many turns and retries.
LiteLLM
Open-source LLM gateway and SDK unifying multiple model providers behind an OpenAI-compatible interface.
Curated Intelligence
AWS Machine Learning BlogT1·Tools
62 pts
AWS Machine Learning BlogT1·Tools
52 pts
An AWS blog post shows how to build a self-service LLM gateway using Amazon Bedrock Application Inference Profiles with LiteLLM on AWS Fargate, abstracting Bedrock behind an OpenAI-compatible API. Developers get a single key to access foundation models, while platform teams gain per-user cost attribution, latency and error observability, and consistent access controls.