Announcing SOCI-indexed images for faster SageMaker AI jobs

Amazon added Seekable OCI (SOCI) indexing to SageMaker AI Training and Processing jobs, letting the service lazy-load container images so jobs start in seconds instead of minutes.

• Deep learning images routinely exceed 10 GB, and downloading them adds minutes to every job that customers pay for as billable job time.
• With SOCI indexing, SageMaker AI fetches only the files needed to start the container and streams the rest in the background as code runs, which Amazon says cut GPU idle time during image pulls by up to 90 percent in its benchmarks.
• Adoption requires pointing to a pre-indexed AWS Deep Learning Container or adding an index to your own image, with no changes to job definitions, entry points or data channels.
• In tests across eight images from 1.8 GB to 35.4 GB, lazy loading held at 3–11 seconds regardless of size, an 8x–46x improvement, and a 13.2 GB image ran about 34x faster across 13 instance type and size combinations.
• Amazon says training performance is unaffected beyond a first-epoch penalty, and jobs fall back to a standard download when no index exists.