From the source
Apple researchers propose split-and-scale, a method that expands the latent capacity of 3D shape representations at inference time while keeping training cost tractable.
- Latent 3D representations encode 3D scenes compactly and support generative models, but the paper says they struggle to capture fine details of complex objects because their latents are capacity-limited.
- Scaling that capacity during training is computationally expensive and often infeasible under memory limits, and simply adding more latents at inference does not use the added capacity well.
- The method treats 3D scenes as continuous functions of 3D coordinates, so a single pretrained tokenizer can be applied at inference to arbitrarily small sub-regions, giving the same latent capacity to a smaller spatial extent.
- The authors ran extensive ablations on the trade-offs between compute, memory, latent capacity and reconstruction quality, and report that their inference-time scaling reaches quality competitive with much larger models that exceed single-GPU memory limits.
- They also trained an autoregressive 3D generator on the representation, which they say performs competitively with state-of-the-art baselines.