# Cartesia — From Tokens to Bytes I: Scaling Hierarchical Networks

- Company: Cartesia (cartesia.ai)
- Announced: 2026-10-28
- Category: research-paper
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://www.cartesia.ai/blog/from-tokens-to-bytes-part-1
- Record: https://forck.live/items/28892-from-tokens-to-bytes-i-scaling-hierarchical-networks
- Subject: Sonic / Ink

Cartesia says its hierarchical networks (H-Nets) can replace hand-designed tokenizers by learning to chunk raw bytes directly inside the model, and it reports large efficiency gains from scaling them. The company says carefully controlled 1-stage H-Nets deliver up to 3-5x compute efficiency gains purely through learned tokenization, measured against a standard scaling ladder for tokenized Transformers. A more flexible 2-stage H-Net, scaled up, outperformed a matched open-source model while using 10x less compute and data, according to Cartesia. The company says H-Nets show behaviors not previously seen in language models, including in-context tokenization, and that ongoing work in biology and vision suggests the architecture can span modalities. Cartesia frames the work as a test of the bitter lesson: tokenization is one of the last hand-designed parts of the language-modeling pipeline, and byte-level models are a way to learn end-to-end from raw data. The results appear in the first of two posts, with a second planned to explore a larger range of models and why H-Nets are more efficient for language modeling.

## Evidence

Verbatim from https://www.cartesia.ai/blog/from-tokens-to-bytes-part-1:

> Since Richard Sutton’s influential essay in 2019, the concept of the bitter lesson has guided much of modern progress in AI. This concept espouses general methods that leverage computation and learn with scale, as free of human-engineered priors as possible. Our interpretation is that, fundamentally, intelligence is about learning from raw data : autonomously finding patterns, building abstractions, and developing meaning from messy low-level inputs, ideally end-to-end and as data-efficiently as possible. As AI advances into complex domains such as science, robotics, and biology – where data is scarce and often difficult for humans to understand, let alone engineer for – internalizing this philosophy to develop more general learning algorithms is more important than ever.
> 
> Language models are perhaps AI’s biggest success story today, and largely embrace the bitter lesson. Their abilities are largely attributable to scaling a few simple algorithms such as the Transformer architecture, next-token prediction, reinforcement learning, and test-time computation.

---

Record: https://forck.live/items/28892-from-tokens-to-bytes-i-scaling-hierarchical-networks
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md