Replicate announces caching of torch.compile artifacts to reduce boot times for models using PyTorch, with specific models starting 2-3x faster and cold boot times reduced by 50-62%.
From the source
We now cache torch.compile artifacts to reduce boot times for models that use PyTorch.
replicate.com
