Hugging Face· PlatformInfrastructure

From Chunks to Blocks: Accelerating Uploads and Downloads on the Hub

Hugging Face's Xet team announces a new chunk-based approach using blocks, shards, and key chunks to accelerate uploads and downloads on the Hub, with open-sourced components xet-core and hf_xet integrated with huggingface_hub.

From the source

To bring faster file transfers to the Hub, we've open-sourced xet-core and hf_xet, an integration with huggingface_hub which uses a chunk-based approach written in Rust.

huggingface.co

Announced

12 Feb 2025 00:00 UTC

Detected

03 Aug 2026 21:12 UTC (+537d after)