From the source
Hugging Face integrates LLM.int8() 8-bit matrix multiplication into its transformers library to reduce memory footprint of large language models without degrading performance.
A Gentle Introduction to 8-bit Matrix Multiplication for transformers at scale using transformers, accelerate and bitsandbytes
Story of the weekCapability change
From the source
we offer LLM.int8() integration for all Hugging Face models
huggingface.co
