From the source
Amazon SageMaker HyperPod now provides Data Capture, a built-in capability that records live inference inputs and outputs to Amazon S3 at three independent tiers, with minimal latency impact and zero impact on model availability.
- The feature addresses a gap for teams running foundation models in production: without a record of production inference data, they cannot detect drift, find failure patterns, or use real interactions to improve the model.
- The walkthrough runs on a single HyperPod cluster, where a home-appliance support router captures its traffic, Athena queries surface a new-product blind spot, a SQL policy labels the captured records, and a fine-tune on that data teaches the router the new product.
- Tiers 1 and 3 both store full payloads, so enabling both stores every record twice; Amazon suggests Tier 1 for compliance and Model Monitor, Tier 2 when only traffic metadata is needed, and Tier 3 when the payloads will be used.
- Because Tier 3 records complete payloads that can include names, addresses and order numbers, Amazon advises protecting that data with sampling, AWS KMS encryption, S3 lifecycle rules and PII redaction before it becomes training data.
- The example uses Qwen2.5-3B-Instruct, which Amazon says is small enough to fine-tune in about a minute and fits on a single L40S, with the inference operator serving on one node and the training operator fine-tuning on another.






