# Amazon — Build efficient agents with knowledge distillation on Amazon SageMaker AI

- Company: Amazon (amazon.com)
- Announced: 2026-11-02T17:01:31+00:00
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://aws.amazon.com/blogs/machine-learning/build-efficient-agents-with-knowledge-distillation-on-amazon-sagemaker-ai/
- Record: https://forck.live/items/30595-build-efficient-agents-with-knowledge-distillation-on-amazon-sagemaker-ai
- Subject: Bedrock / Nova

Amazon's guide shows how to cut the cost of agentic assistants by distilling a frontier teacher model into a smaller student on Amazon SageMaker AI. An agentic assistant is not one model call but a loop of search, read, refine, verify and decide, and each step is a separate billed call to the frontier model, so per-task cost compounds at production scale. The post walks through sequence-level knowledge distillation (SeqKD), which trains a student on the teacher's own text outputs with supervised fine-tuning and needs no teacher logits or reinforcement-learning infrastructure. Because SeqKD only needs the teacher's text, it works with a closed or API-only teacher, unlike logit-based or on-policy methods that require internal access. The guide covers generating multi-step trajectories with a frontier teacher, keeping only the runs that solve the task, encoding them for supervised fine-tuning, running training on serverless infrastructure, and evaluating the student against a held-out benchmark. The post cites the DeepSeek-R1 distilled models, whose teacher reasoning was distilled into smaller Qwen and Llama students, as a well-known example of the approach.

## Evidence

Verbatim from https://aws.amazon.com/blogs/machine-learning/build-efficient-agents-with-knowledge-distillation-on-amazon-sagemaker-ai/:

> You built a shopping assistant on a frontier model, and it works. A shopper asks for “a black waist bag under twenty dollars from a shop with at least 4.5 stars,” and the agent searches the catalog, inspects candidates, checks the constraints, and commits to a recommendation. But what does that interaction actually cost? An agentic assistant is not one model call, it is a loop: search, read results, refine, verify, decide. A single completed task can be multiple calls to that frontier model, and you pay for every one, on every shopper. At production scale, that per-task cost compounds, erodes the project’s ROI, and stops the project from shipping.
> 
> One way to make an agentic application more efficient is to use a smaller model, which is faster to respond and more cost-effective to host. The catch is quality: a smaller model usually falls short on the task, which is why the larger one was chosen in the first place. Knowledge distillation closes that gap by transferring the behavior of the larger “teacher” model into a smaller “student”, training the student to reproduce the teacher’s outputs, so it keeps much of the quality at the smaller model’s speed and cost.

---

Record: https://forck.live/items/30595-build-efficient-agents-with-knowledge-distillation-on-amazon-sagemaker-ai
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md