From the source
Starting today, Psyche will train a number of new models in parallel, all aimed at creating useful, novel open source AI .
- Our initial training run on testnet verified that we can train models over internet bandwidth at significant parameter and dataset sizes.
- By both measures, our Consilience 40B run was the largest distributed pre-training run ever .
- Though we have hit some important milestones with this run, more than anything Nous strives to produce high quality intelligence.
- While we're happy to have reached this milestone for provability, Nous has one mission: to open-source and distribute the world's best intelligence.
- In that spirit, we're moving from provability to performance , as we begin our efforts to pre-train, post-train, and RL, state-of-the-art open source models.
- Since the launch of the Psyche testnet we've made substantial improvements to the codebase, including full trainer abstraction , which will allow us to train arbitrary models and move beyond pre-training to supervised fine-tuning (and eventually reinforcement learning).
- …