From the source
Hugging Face published a blog post comparing the bitsandbytes and auto-gptq quantization schemes supported natively in Transformers, discussing their benefits and drawbacks.
Overview of natively supported quantization schemes in 🤗 Transformers
From the source
We aim to give a clear overview of the pros and cons of each quantization scheme supported in transformers to help you decide which one you should go for.
huggingface.co
