# Apple — Pixels to Tokens: Token Space Efficient Active Learning for Low-Budget Semantic Segmentation

- Company: Apple (apple.com)
- Announced: 2026-10-21
- Category: research-paper
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://machinelearning.apple.com/research/pixels-tokens-active-learning
- Record: https://forck.live/items/34331-pixels-to-tokens-token-space-efficient-active-learning-for-low-budget-semantic
- Subject: Machine Learning Research

Apple researchers, with the University of Waterloo, introduced TEAL, a pixel-level active learning framework that selects which pixels to label using a frozen vision transformer's own token grid. Existing active learning for vision transformers works at the patch or region level, so annotation costs stay high, and extending it to one pixel per image means a method must find informative locations and train a decoder from a very sparse signal. TEAL applies a two-level MaxHerding strategy over multi-layer token descriptors to pick representative candidates, then refines them with margin-based uncertainty on a finer decoder grid, avoiding artifacts from interpolating token embeddings to dense pixel resolutions. Across CamVid, Cityscapes, ADE20K and Pascal Context, the authors report gains of up to +24.36mIoU over the strongest baseline with a single labeled pixel per image, and up to 97% of fully supervised performance with 10 px. The improvements held when all baselines used the same DINOv3 backbone, which the authors say shows token-space selection combined with decoder-grid uncertainty refinement makes ViT token geometry effective for very low-budget active segmentation.

## Evidence

Verbatim from https://machinelearning.apple.com/research/pixels-tokens-active-learning:

> Authors Utku Cicek†, Mahip Singh†, Ziyao Shang†, Kimathi Kaai, Pedro Pablo Guerrero Vela, C Thomas, Alexander Wong, Sirisha Rambhatla†
> 
> While foundation models and vision transformers (ViTs) have redefined visual representations, current active learning (AL) strategies for these backbones largely operate at the patch or region level, where annotation requirements remain high due to extensive pixel labeling. Extending ViT-based AL to the pixel-level, low-budget regime is uniquely challenging: with as little as one pixel query per image, a method must simultaneously identify the most informative locations and train a reliable decoder head from an extremely sparse signal. We introduce TEAL (Token-space Efficient Active Learning), the first pixel-level AL framework built around ViT representations. TEAL uses a frozen ViT backbone and performs diversity-based selection directly on its native token lattice, avoiding the artifacts introduced by interpolating token embeddings to dense pixel resolutions.

---

Record: https://forck.live/items/34331-pixels-to-tokens-token-space-efficient-active-learning-for-low-budget-semantic
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md
