# Sakana AI — Sparser, Faster, Lighter Transformer Language Models

- Company: Sakana AI (sakana.ai)
- Announced: 2026-05-08T15:00:00+00:00
- Category: research-paper
- Subject: Sakana models
- Source: https://sakana.ai/twell
- Record: https://forck.live/items/4684-sparser-faster-lighter-transformer-language-models

Sakana AI, in collaboration with NVIDIA, announces a paper accepted at ICML 2026 introducing open-source GPU kernels and a new sparse packing format (TwELL) to speed up inference and training of sparse transformer language models, achieving >20% speedups and reduced memory usage.

## Evidence

Verbatim from https://sakana.ai/twell:

> Excited to share our new #ICML2026 paper in collaboration with NVIDIA: "Sparser, Faster, Lighter Transformer Language Models". This work introduces new open-source GPU kernels and data formats for faster inference and training of sparse transformer language models:

---

Record: https://forck.live/items/4684-sparser-faster-lighter-transformer-language-models
Catalogue: https://forck.live/llms.txt
Feed: https://forck.live/feed.md