Optimizing On-Device Inference for Apple Silicon

Perplexity released Lily, a lightweight local inference engine built specifically for Apple silicon and the Qwen3.6-35B-A3B model, with separate optimizations for prefill and decode phases.

• Benchmarked on an M5 Max MacBook Pro, Lily achieves 1.23× prefill throughput and 1.35× decode throughput compared to MLX-LM, reaching 5,749.9 prefill tokens per second and 186.6 decode tokens per second at 4K-token prompt and context lengths.
• A standalone demo is publicly available on GitHub.