# Amazon — Using NVIDIA Run:ai GPU Fractioning on Amazon EKS

- Company: Amazon (amazon.com)
- Announced: 2026-11-19T16:13:51+00:00
- Category: not stated
- Coverage: not counted
- Announcement: no
- Group: routine
- Source: https://aws.amazon.com/blogs/machine-learning/using-nvidia-runai-gpu-fractioning-on-amazon-eks/
- Record: https://forck.live/items/38709-using-nvidia-run-ai-gpu-fractioning-on-amazon-eks
- Subject: Bedrock / Nova

Amazon's walkthrough shows how to build a custom Amazon Linux 2023 image and configure NVIDIA Run:ai GPU Fractioning on Amazon EKS, giving each Pod an enforced GPU-memory contract on a single A10G. The post addresses teams whose notebooks, model evaluations and small inference services wait for GPUs even when memory remains free, because the NVIDIA device plugin exposes the whole GPU as one nvidia.com/gpu resource with sharing disabled. In the validated G5/A10G example, two Pods each hold an 8 GiB memory limit and allocate 6 GiB, and the walkthrough verifies per-Pod metrics and rejection of a 9 GiB allocation. NVIDIA announced the open source release of NVIDIA Run:ai on October 13, 2026, and the example uses two of its four modular components, KAI Scheduler and the GPU Fractioning Operator. The approach applies to NVIDIA GPUs meeting the operator's hardware and software requirements, including GPUs without MIG, and the post says other configurations need compatibility checks and validation. The post states that the demonstrated density improvement does not establish a throughput gain or a universal cost reduction, and that a memory limit does not reserve proportional compute.

## Evidence

Verbatim from https://aws.amazon.com/blogs/machine-learning/using-nvidia-runai-gpu-fractioning-on-amazon-eks/:

> GPU sharing is not new: Multi-Instance GPU (MIG), time-slicing, and Multi-Process Service (MPS) address different isolation and utilization needs. NVIDIA Run:ai GPU Fractioning builds on MPS with Kubernetes-integrated scheduling, per-Pod memory contracts, enforcement, and metrics.
> 
> NVIDIA Run:ai is AI workload orchestration software for Kubernetes. Its GPU Fractioning Operator can help several teams share one or two GPUs on Amazon Elastic Kubernetes Service (Amazon EKS) . Notebooks, model evaluations, and small inference services can leave teams waiting even when GPU memory remains available. In this walkthrough’s baseline, the NVIDIA device plugin exposes the single GPU as one nvidia.com/gpu resource, with GPU sharing disabled. Two Pods each requesting nvidia.com/gpu: 1 cannot run concurrently on that node, even when their combined GPU-memory demand fits.
> 
> NVIDIA Run:ai GPU Fractioning adds Kubernetes scheduling integration, enforced GPU-memory limits, and per-Pod metrics to software-based sharing. The approach applies to NVIDIA GPUs that meet the GPU Fractioning Operator’s hardware and software requirements, including GPUs without MIG.

---

Record: https://forck.live/items/38709-using-nvidia-run-ai-gpu-fractioning-on-amazon-eks
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md
