From the source
Apple researchers, with the University of Waterloo, introduced TEAL, a pixel-level active learning framework that selects which pixels to label using a frozen vision transformer's own token grid.
- Existing active learning for vision transformers works at the patch or region level, so annotation costs stay high, and extending it to one pixel per image means a method must find informative locations and train a decoder from a very sparse signal.
- TEAL applies a two-level MaxHerding strategy over multi-layer token descriptors to pick representative candidates, then refines them with margin-based uncertainty on a finer decoder grid, avoiding artifacts from interpolating token embeddings to dense pixel resolutions.
- Across CamVid, Cityscapes, ADE20K and Pascal Context, the authors report gains of up to +24.36mIoU over the strongest baseline with a single labeled pixel per image, and up to 97% of fully supervised performance with 10 px.
- The improvements held when all baselines used the same DINOv3 backbone, which the authors say shows token-space selection combined with decoder-grid uncertainty refinement makes ViT token geometry effective for very low-budget active segmentation.