# Amazon — Cut generative AI costs: Code for rules, models for judgment

- Company: Amazon (amazon.com)
- Announced: 2026-11-17T21:14:55+00:00
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://aws.amazon.com/blogs/machine-learning/cut-generative-ai-costs-code-for-rules-models-for-judgment/
- Record: https://forck.live/items/37570-cut-generative-ai-costs-code-for-rules-models-for-judgment
- Subject: Bedrock / Nova
- Models affected: Strands Decider
- License: Apache License 2.0

Amazon's guide tells teams to decide whether each workflow step needs a model at all, since routing every task through an LLM makes small calls a large production line item. The post says unneeded calls cost in three ways: spend grows with request count, latency stacks as each call adds hundreds of milliseconds, and output varies for the same input. It walks through three ordered questions, stopping at the first that meets the step's requirements: can a rule check success, can a model write the rule once, and does the step pick one answer from a fixed list. Steps that fail all three are open-ended and run on a right-sized model on Amazon Bedrock, while rule-shaped tasks should stay in deterministic code. For routing and classification, the post lists four options: a small Bedrock model with structured outputs, embedding similarity, a trained classifier on Amazon SageMaker AI, or Strands Decider. Strands Decider is an experimental open source decision model under the Apache License 2.0 that returns a calibrated confidence with each answer, and its README reports answers at a confidence of 0.9 or more are right about 95 percent of the time on short classification tasks. Because it is experimental, the post says to validate that result on your own data before production use, and to check any escalation signal against labeled data.

## Evidence

Verbatim from https://aws.amazon.com/blogs/machine-learning/cut-generative-ai-costs-code-for-rules-models-for-judgment/:

> The biggest cost lever in a generative AI system is whether you call a model at all. We often see teams route every task through a large language model (LLM), including input validation, formatting, and routing. Each call is cheap in isolation, but at production scale these small costs become one of the largest line items in the architecture. Wherever a deterministic rule can do the job, skipping the model saves money, cuts latency, and removes a source of non-determinism. In this post, you learn how to decide whether each step in a workflow needs a model at all. You can then map each option to AWS: deterministic code, a constrained decision call, or a right-sized model on Amazon Bedrock .
> 
> Why unneeded model calls add up
> 
> The pattern usually starts in a prototype. A model call works for one step, so it becomes the default for the next. Those steps include validating input, reformatting a request body, and picking a route. At prototype volume, nobody notices the cost.
> 
> At production volume, unneeded model calls cost you in three ways:
> 
> Spend grows with request count, even for low-value steps.
> 
> Latency stacks, because every model call in a chain adds hundreds of milliseconds.

---

Record: https://forck.live/items/37570-cut-generative-ai-costs-code-for-rules-models-for-judgment
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md
