From the source
Apertus LLM Family Expansion via Distillation and Quantization We release a set of 16 small models that we are calling the Apertus Mini collection , based on distillation and quantization techniques applied to the original Apertus v1 large language model.
Base and instruct models are available in the following new sizes, along with 10 other quantization levels: Apertus 1.1 0.5B and 0.5B Instruct (500 million parameters) Apertus 1.1 1.5B and 1.5B Instruct (1.5 billion parameters) Apertus 1.1 4.0B and 4.0B Instruct (4 billion parameters) These builds, created on the same infrastructure as our initial release, have been optimized for use in memory- or compute-constrained AI systems, such as portable or embedded devices.
A linked technical report, being shared in an upcoming ICML workshop, describes the process used and evaluates the performance across a series of benchmarks.
Downloads and an online demo are available, keep reading for background details.
Details The training pipeline starts by very efficient distillation from the larger Apertus v1 8B teacher model, where we have recorded prediction logits for each next tokens, and wrote them to disk.
…
