Now in Nature: Retrofitting language models to operate over bytes

Ai2's research on byteifying language models, published in Nature, demonstrates converting subword models into byte-level models with additional training.

The company released new checkpoints extending the approach to Qwen 3 8B and Llama 3 8B, creating Bwen 8B and Blama 8B, and also released Stage 1 checkpoints that keep the original model frozen while training byte-level components.