Amp announced that response times of Amp Tab, its in-editor completion engine, are now 30% faster, with up to 50% improvements during peak usage.
The improvement was achieved through infrastructure optimization with 
Baseten, switching to TensorRT-LLM as the inference engine, implementing KV caching with speculative decoding, and using a modified lookahead decoding algorithm with improved n-gram candidate selection and variable-length speculations.
