From the source
Lead story
Ask your AI
Top stories
Models & availability
Latest
Amazon· Bedrock / NovaReduce inference cold starts on Amazon SageMaker HyperPod with model caching
Model caching reduces SageMaker HyperPod inference cold starts from 25-30+ minutes to seconds.
Amazon launched model caching for Amazon SageMaker Inference on HyperPod, which pre-loads model weights and container images onto cluster nodes to reduce inference cold starts from tens of minutes to seconds.
From the source
Today we’re launching model caching for Amazon SageMaker Inference on HyperPod. Model caching pre-loads model weights and container images onto cluster nodes before pods need them.
aws.amazon.com
Other stories
Amazon · 2 OctSweep thousands of leases for compliance using Amazon Quick and the Adjudicated Query patternAmazon · 2 OctAdd secure Web Search to Claude Desktop with Amazon Bedrock AgentCoreAmazon · 2 OctFine-tune a search agent with multi-turn RL on Amazon SageMaker AI