Toolsslug: sagemaker-hyperpod

SageMaker HyperPod

AWS managed cluster service for large-scale model training and inference with elastic GPU capacity and resilient scheduling.

Curated Intelligence

4 items · Page 1/1
AWS Machine Learning BlogT1·Infra
58 pts
AWS Launches SageMaker HyperPod Global Inference Gateway

AWS introduced the SageMaker HyperPod Global Inference Gateway, extending its GPU-aware routing from a single cluster to a single managed endpoint spanning multiple clusters and Regions. The gateway selects clusters using real-time signals such as KV cache utilization and queue depth, carries requests over the AWS backbone, and provides automatic failover with SigV4 and IAM authentication. It targets capacity waste when one cluster queues requests while GPUs elsewhere sit idle.

AWS Machine Learning BlogT1·Infra
58 pts
SageMaker Inference 2026 Update Adds Cross-Region Routing and GPU Sharing

AWS reviewed its 2026 SageMaker Inference updates, including new routing strategies for HyperPod Inference, broader model-serving framework support and expanded multi-Region networking, with the Inference Gateway handling routing and managed NVIDIA Dynamo added as a new serving framework. For managed endpoints, inference recommendations now run from SageMaker AI Studio, the Python SDK and coding agents, and endpoints can route requests across Regions and accounts, helping teams re-select configurations after traffic shifts, pack more models onto paid-for GPUs and fix idle capacity across Regions.

AWS Machine Learning BlogT1·Infra
62 pts
Amazon SageMaker HyperPod Adds Managed NVIDIA Dynamo Inference

Amazon SageMaker HyperPod Inference now supports managed NVIDIA Dynamo, pairing Dynamo's distributed inference with HyperPod's infrastructure management to address single-instance GPU memory limits and long-prompt slowdowns. The integration enables multi-node deployment, disaggregated prefill and decode, and KV-aware routing across engines such as vLLM, SGLang, and TensorRT-LLM, backed by EFA networking, coordinated GPU scheduling, and node health monitoring.

AWS Machine Learning BlogT1·Tools
42 pts
SageMaker HyperPod Adds Data Capture for Closed-Loop Fine-Tuning

AWS has added built-in Data Capture to SageMaker HyperPod, recording live inference inputs and outputs to S3 across three independent tiers with zero impact on model availability. A home-appliance support router walkthrough shows the full flywheel: capture traffic, surface a new-product blind spot via Athena, label records with SQL, then fine-tune and measure accuracy before and after.