# Amazon — Route coding agent requests to the right model on Amazon Bedrock with NVIDIA NeMo Switchyard

- Company: Amazon (amazon.com)
- Announced: 2026-11-13T22:30:12+00:00
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://aws.amazon.com/blogs/machine-learning/route-coding-agent-requests-to-the-right-model-on-amazon-bedrock-with-nvidia-nemo-switchyard/
- Record: https://forck.live/items/36107-route-coding-agent-requests-to-the-right-model-on-amazon-bedrock-with-nvidia
- Subject: Bedrock / Nova
- Models affected: Claude Opus 5.5, NVIDIA Nemotron 3 Super 120B

Amazon published a walkthrough for deploying LiteLLM with NVIDIA's open source NeMo Switchyard routing plugin on Amazon Bedrock, so teams can send each coding agent request to a different model. The post argues that price per token does not reveal what a whole coding task costs, because the total depends on how many turns a model needs, how much history each turn replays, and how often a task is retried or never finishes. It recommends judging routing by cost per solved task, the total cost of runs divided by the number of runs whose tests pass. The sample deploys LiteLLM and the Switchyard plugin in one container on Amazon ECS on AWS Fargate in private subnets, with AWS WAF filtering requests and an Application Load Balancer forwarding only three paths. The integration supports Stage routing, which reads recent tool activity and results in the request history and picks between two candidate models, and Random routing, which picks from configured models with optional weights for traffic splits and A/B comparisons. Neither strategy calls a judge model, and classifier-backed strategies such as Task and Composite routing need intermediate model calls the integration does not support. The sample installs switchyard-server 0.3.0 and runs the published LiteLLM v1.102.0 image, and NVIDIA documents switchyard-server 0.3.0 as a demo and evaluation component whose configuration can change between releases.

## Evidence

Verbatim from https://aws.amazon.com/blogs/machine-learning/route-coding-agent-requests-to-the-right-model-on-amazon-bedrock-with-nvidia-nemo-switchyard/:

> Coding agents such as Claude Code and OpenAI Codex CLI work in tasks. A developer asks for a change, and the agent reads files, edits code, and runs tests until the work is done. One task can take a few model calls or dozens. What the developer pays for is the task, not one call.
> 
> The price per token alone doesn’t tell you what a whole task costs. The total depends on how many turns a model needs, how much history each turn replays, and how often a task has to be retried or never finishes. You pay for every attempt, including attempts on tasks that never finish. So when you route a coding agent’s requests between models, judge the routing by cost per solved task: the total cost of your runs divided by the number of runs whose tests pass.
> 
> NVIDIA NeMo Switchyard is an open source library that helps an AI agent choose which model handles each request. You define the models that it can use and the strategy that it follows. Its LiteLLM integration supports two strategies. Stage routing reads the recent tool activity and results in the request history, and then picks one of two candidate models.

---

Record: https://forck.live/items/36107-route-coding-agent-requests-to-the-right-model-on-amazon-bedrock-with-nvidia
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md
