Parcae: Doing more with fewer parameters using stable looped models
New model
Together AI introduces Parcae, a stable looped language model architecture that achieves the quality of a Transformer with twice the parameters, using fewer parameters by increasing recurrence.
A 770M parameter Parcae matches a 1.3B parameter Transformer.
