CRoSS: A Continual Robotic Simulation Suite for Scalable Reinforcement Learning with High Task Diversity and Realistic Physics Simulation

New Benchmark Suite CRoSS Enhances Continual Reinforcement Learning for Robotics

A groundbreaking benchmark suite called Continual Robotic Simulation Suite (CRoSS) has been introduced to advance continual reinforcement learning (CRL) by addressing the challenge of agents learning from a sequence of tasks without forgetting previously acquired policies. Developed using the Gazebo simulator, CRoSS facilitates research in robotic settings with high physical realism.

CRoSS utilizes two distinct robotic platforms: a two-wheeled differential-drive robot and a seven-joint robotic arm. The differential-drive robot navigates various scenarios, including line-following and object-pushing tasks, using lidar, camera, and bumper sensors. The robotic arm focuses on goal-reaching tasks, offering high-level Cartesian control and low-level joint angle control. CRoSS also introduces kinematics-only variants for the robotic arm, enabling simulations to run significantly faster when physical sensor readings are not required.

Extensibility and Reproducibility

CRoSS is designed with extensibility in mind, allowing researchers to incorporate a wide range of simulated sensors into their studies. To enhance reproducibility, the suite includes a containerized setup using Apptainer, ensuring users can run the benchmark without extensive configuration.

The performance of standard reinforcement learning algorithms, such as Deep Q-Networks (DQN) and policy gradient methods, has been reported within the suite, illustrating its efficacy as a scalable benchmark for CRL research. The introduction of CRoSS marks a significant step forward in developing sophisticated robotic learning systems.