# Amazon — Deploy decision models with llama.cpp on Amazon SageMaker AI

- Company: Amazon (amazon.com)
- Announced: 2026-10-12T15:51:46+00:00
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://aws.amazon.com/blogs/machine-learning/deploy-decision-models-with-llama-cpp-on-amazon-sagemaker-ai/
- Record: https://forck.live/items/19906-deploy-decision-models-with-llama-cpp-on-amazon-sagemaker-ai
- Subject: Bedrock / Nova

Amazon published a guide on deploying decision models using llama.cpp on Amazon SageMaker AI. The post covers two deployment paths: a standard text-generation path using the AWS DLC for llama.cpp with an OpenAI-compatible API, and a structured-decision path using a SemIf adapter that returns probabilities for declared options. The guide includes CPU and GPU deployment options and benchmarks showing GPU endpoints were faster than CPU.

## Evidence

Verbatim from https://aws.amazon.com/blogs/machine-learning/deploy-decision-models-with-llama-cpp-on-amazon-sagemaker-ai/:

> The structured-decision path builds a child image that replaces the default server with a SemIf adapter and returns probabilities for declared options.

---

Record: https://forck.live/items/19906-deploy-decision-models-with-llama-cpp-on-amazon-sagemaker-ai
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md