From the source
Amazon SageMaker AI now offers cross-Region inference routing, letting applications invoke one router path across endpoints in multiple AWS Regions and accounts instead of building failover code themselves.
- Until now, teams serving models on SageMaker AI managed endpoints had to write their own logic to detect a struggling endpoint, switch Regions, decide which errors were safe to retry, and move traffic back without flapping.
- The new inference router is a Regional resource that holds routing targets, each naming a logical model, and destinations registered by ARN that identify their Region and account.
- The router prefers destinations in its own Region, spreads requests evenly across healthy local destinations, and sends traffic elsewhere only when local ones cannot serve reliably.
- SageMaker AI scores each destination's health from recent requests in its own Region, where server errors, throttles and timeouts lower the score and platform errors lower it most, while request-caused errors such as invalid input do not.
- Health is published to Amazon CloudWatch as healthy, overloaded or degraded, and SageMaker AI manages failover thresholds and timing so customers do not tune them.
- The router does not create capacity, scale endpoints, deploy models or copy artifacts, so each destination must already be running in its Region.






