SageMaker AI Launches Capacity-Aware Inference — Automatic Instance Fallback When GPU Capacity Is Constrained

AWS Machine Learning Blog·medium signal

Amazon SageMaker AI now lets you define a prioritized list of instance types for inference endpoints, automatically falling back through the list when capacity is constrained during creation, scale-out, or scale-in. Previously, if your specified GPU instance type had no capacity, the endpoint would fail or the autoscaler would retry indefinitely while traffic grew. Works with single-model, inference-component, and async endpoints.