From the source
The implementation of KV caching from scratch in the nanoVLM repository, a small pure PyTorch codebase for training Vision Language Models.
It reports a 38% speedup in generation and discusses the theory behind KV caching and its practical application.
