# Hugging Face — Creating custom kernels for the AMD MI300

- Company: Hugging Face (huggingface.co)
- Announced: 2025-07-09T00:00:00+00:00
- Category: developer-tool-release
- Subject: Platform
- Models affected: Llama 3.1 405B
- Source: https://huggingface.co/blog/mi300kernels
- Record: https://forck.live/items/1664-creating-custom-kernels-for-the-amd-mi300

Hugging Face and AMD collaborated to release open-source custom kernels optimized for the AMD MI300X GPU, achieving significant speedups for serving Llama 3.1 405B in FP8 on VLLM. The kernels include a fused residual connection, RMS norm and FP8 conversion kernel, a fused SwiGLU activation and FP8 conversion kernel, and a skinny GEMM kernel. The kernels are available in the hf-rocm-kernels repository.

## Evidence

Verbatim from https://huggingface.co/blog/mi300kernels:

> Hugging Face collaborated with AMD to deliver state-of-the-art performance on AMD platforms and make it benefit the open source community. As part of this partnership, we decided with AMD to focus on delivering open-source optimized kernels to improve the performance of serving Llama 3.1 405B in FP8 on a node of 8 MI300X using VLLM.

---

Record: https://forck.live/items/1664-creating-custom-kernels-for-the-amd-mi300
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md