Creating custom kernels for the AMD MI300

Hugging Face and AMD collaborated to release open-source custom kernels optimized for the AMD MI300X GPU, achieving significant speedups for serving Llama 3.1 405B in FP8 on VLLM.

The kernels include a fused residual connection, RMS norm and FP8 conversion kernel, a fused SwiGLU activation and FP8 conversion kernel, and a skinny GEMM kernel.

The kernels are available in the hf-rocm-kernels repository.