# Amazon — Keep inference running through regional capacity shortfalls with cross-Region inference routing in Amazon SageMaker AI

- Company: Amazon (amazon.com)
- Announced: 2026-11-19T18:20:54+00:00
- Category: capability-change
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://aws.amazon.com/blogs/machine-learning/keep-inference-running-through-regional-capacity-shortfalls-with-cross-region-inference-routing-in-amazon-sagemaker-ai/
- Record: https://forck.live/items/38815-keep-inference-running-through-regional-capacity-shortfalls-with-cross-region
- Subject: Bedrock / Nova

Amazon SageMaker AI now offers cross-Region inference routing, letting applications invoke one router path across endpoints in multiple AWS Regions and accounts instead of building failover code themselves. Until now, teams serving models on SageMaker AI managed endpoints had to write their own logic to detect a struggling endpoint, switch Regions, decide which errors were safe to retry, and move traffic back without flapping. The new inference router is a Regional resource that holds routing targets, each naming a logical model, and destinations registered by ARN that identify their Region and account. The router prefers destinations in its own Region, spreads requests evenly across healthy local destinations, and sends traffic elsewhere only when local ones cannot serve reliably. SageMaker AI scores each destination's health from recent requests in its own Region, where server errors, throttles and timeouts lower the score and platform errors lower it most, while request-caused errors such as invalid input do not. Health is published to Amazon CloudWatch as healthy, overloaded or degraded, and SageMaker AI manages failover thresholds and timing so customers do not tune them. The router does not create capacity, scale endpoints, deploy models or copy artifacts, so each destination must already be running in its Region.

## Evidence

Verbatim from https://aws.amazon.com/blogs/machine-learning/keep-inference-running-through-regional-capacity-shortfalls-with-cross-region-inference-routing-in-amazon-sagemaker-ai/:

> Serving a model from more than one AWS Region is a well-established pattern for highly available inference: when one Region runs short of GPU capacity, requests move to another. The pattern is simple to describe and hard to operate. Until now, if you served models on Amazon SageMaker AI managed endpoints, cross-Region routing was a piece you still had to build yourself.
> 
> Today, we’re introducing cross-Region inference routing in Amazon SageMaker AI. It uses a new resource, the inference router, that gives your application one invocation path across the endpoints and inference components you already run, in one or more AWS Regions and AWS accounts. You register the destinations that serve a model, and your application invokes the router instead of a specific endpoint. The router keeps requests in its own Region while the local destination is healthy, shifts new requests to another registered destination when it isn’t, and returns traffic gradually after recovery.
> 
> In this post, we explain how cross-Region inference routing decides where each request runs, walk through an example with one model in two Regions, and cover monitoring, limits, and what you pay.

---

Record: https://forck.live/items/38815-keep-inference-running-through-regional-capacity-shortfalls-with-cross-region
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md
