From the source
Ai2 is introducing FoundationWorlds, a framework for building controlled simulated environments to study robot learning, with ToyBlocks as the first world.
- ToyBlocks is an open simulation framework built around a robot arm and blocks on a tabletop, where the robot must reason about how blocks interact while pushing, flipping and stacking them.
- The setting is deliberately simple, but Ai2 says its narrow scope makes it possible to meet four requirements: realistic physics and appearance, automatic generation of tasks and demonstrations at scale, challenges that combine reasoning with dexterity, and broad coverage of the tasks possible within a domain.
- ToyBlocks automatically generates demonstrations without teleoperation, and researchers can vary training mixtures, for example generating more pushing or flipping demonstrations and evaluating on tasks kept out of training.
- Skill training runs in Isaac Lab,
Nvidia's open-source framework for robot learning in simulation, using reinforcement learning and motion planning to learn non-prehensile skills such as pushing a block or flipping it onto another face.
- A task-and-motion planner sequences skills and plans movements, and ToyBlocks generates and executes hundreds of plans in parallel on a single GPU, reaching up to 24 times real-time performance.
- Ai2 says it plans to extend the approach to increasingly complex environments beyond ToyBlocks.






