Amazon· Bedrock / NovaCapability change

Introducing new Ray capabilities on SageMaker HyperPod

Amazon announces new Ray capabilities on SageMaker HyperPod, integrating Ray with HyperPod's purpose-built infrastructure for foundation model training and serving. Data scientists can create Ray clusters, open Ray Dashboard and Grafana dashboards, connect JupyterLab or Code Editor workspaces, submit distributed jobs, and configure hung job detection from SageMaker Studio. The integration provides automatic fault tolerance, tiered checkpointing, and SageMaker JumpStart integration for loading model weights directly into Ray Serve endpoints.

From the source

With this launch, data scientists can create Ray clusters, open the Ray Dashboard and Amazon Managed Grafana observability dashboards, connect a JupyterLab or Code Editor workspace to their cluster, submit distributed jobs, and configure hung job detection, all from SageMaker Studio . At the application level, Ray training jobs gain automatic fault tolerance through HyperPod node health monitoring and recovery, plus tiered checkpointing for faster resume through HyperPod distributed tiered storage. SageMaker JumpStart integration loads model weights directly into Ray Serve endpoints, with KV cache offloading to tiered storage for serving long-context requests.

aws.amazon.com