tinker-atropos: An Integration Layer for the Tinker API and Atropos RL Framework

The team at Nous Research proudly presents tinker-atropos , our integration layer between the Tinker API and our Atropos reinforcement learning (RL) framework.

• We designed Atropos to be a fully decoupled environment service, separating concerns around trainer management, rollout collection, and environment setup into three distinct components.
• Consequently, we found Tinker to be a perfect fit for this paradigm, providing easier management of the inference weights while remaining as close to a fully on-policy approach as possible.
• The goal of our integration layer was to enable any Atropos environment to plug into Tinker with minimal, if any, modifications needed.
• We hope this effort will introduce new Atropos users to the Tinker framework and allow Tinker users to immediately get started with Atropos environments.
• Through decoupling training, inference, and trajectory collection, we are able to present a seamless, easy-to-use integration layer between the Tinker API and the Atropos RL framework.
• Technical Approach / Architecture Overview To integrate with Tinker, we first required an inference layer that could communicate with the SamplingClient .
• …