# Amazon — Amazon SageMaker Inference: 2026 launches in review, part 2

- Company: Amazon (amazon.com)
- Announced: 2026-11-20T00:34:07+00:00
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-inference-2026-launches-in-review-part-2/
- Record: https://forck.live/items/39012-amazon-sagemaker-inference-2026-launches-in-review-part-2
- Subject: Bedrock / Nova

Amazon's second 2026 SageMaker Inference retrospective walks through five stages of running models in production, following one hypothetical team from a single 70B-parameter support assistant to a larger reasoning model across two Regions. Inference recommendations, launched in April as an API, gained four updates that put it in SageMaker AI Studio, the SageMaker Python SDK v3, and coding agents. The Studio workflow, added in July 2026, lets teams pick a use-case profile and an optimization goal, then benchmarks candidate configurations on real GPU infrastructure using NVIDIA AIPerf and ranks them by TTFT, inter-token latency, throughput, and cost. The SDK integration, in version 3.17.0 and later, benchmarks a live endpoint against production traffic and deploys the top-ranked configuration in a single call, and supports parallel jobs comparing serving frameworks such as LMI and vLLM. September 2026 added LoRA adapter support, accepting adapters as Model Package ARNs or S3 URIs and deploying them as Inference Components so adapters can change without disrupting the base model. A second September update added an SLA-based concurrency search that finds the highest concurrency an endpoint can sustain without breaching latency or error targets.

## Evidence

Verbatim from https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-inference-2026-launches-in-review-part-2/:

> Once a model is in production, the questions change. Teams want to know which configuration the model should run on now that traffic is real, how to fit more models onto GPUs they already pay for, what to do when the next model is too large for one machine, why requests wait in one Region while GPUs sit idle in another, and why latency rose when nobody changed anything. This post takes those questions in the order teams usually meet them.
> 
> For HyperPod Inference, it listed additional routing strategies, broader support for model serving frameworks, and expanded multi-Region networking. The Inference Gateway delivers the routing, and managed NVIDIA Dynamo is a new serving framework. For managed endpoints, the roadmap pointed to deeper integration with the rest of the AWS AI landscape. Inference recommendations now run from Amazon SageMaker AI Studio, the SageMaker Python SDK, and the coding agents engineers already use, and endpoints can now route requests across Regions and accounts.
> 
> To keep each section concrete, we follow one hypothetical team through the blog post.

---

Record: https://forck.live/items/39012-amazon-sagemaker-inference-2026-launches-in-review-part-2
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md
