How to use

This section provides instructions on how to use the ROS Gazebo Gym framework to train reinforcement learning algorithms.

General usage instructions

The ROS Gazebo Gym framework contains several gymnasium robotics task environments that can be used to train RL algorithms. Please see the environments section for more information about all available task environments. All the environments in the ROS Gazebo Gym framework can be imported like any other gymnasium environment. The environments will be registered under the gymnasium namespace when imported from the ros_gazebo_gym package or any of its submodules. You can also prefix the environment name with the ros_gazebo_gym module name prefix, separated by a colon, to use it directly inside the gym.make function (e.g. ros_gazeb_gym:PandaReach-v1).

Important

Since the task environments found in the ROS Gazebo Gym are created in Gazebo and ROS, you have to make sure that you first build and source the ros_gazebo_gym catkin workspace before importing any task environments.

Usage Examples

A collection of usage examples showcasing how to utilize the ROS Gazebo Gym framework can be discovered in the ros_gazebo_gym_examples package. These illustrative examples are tailored for integration with the stable-baselines3 package. The current selection of examples includes:

To embark on a training session for a reaching task involving a (simulated) Panda Emika Franka robot using the Stable Baselines SAC algorithm, follow these steps:

  1. Clone the ros_gazebo_gym_examples package within the Catkin workspace established for the ros_gazebo_gym package.

  2. Complete the installation of all requisite ROS dependencies and perform the catkin workspace reconstruction (refer to Install system dependencies and Build the package).

  3. Source the Catkin workspace (e.g., execute . ../develop/setup.bash).

  4. Launch any of the examples from the ros_gazebo_gym_examples package using the roslaunch command.

Here’s a demonstration of how to launch the training example:

roslaunch ros_gazebo_gym_examples start_panda_training.launch

Note

To streamline the installation process, the ros-gazebo-gym-ws repository is provided. This repository encompasses both the ros_gazebo_gym and ros_gazebo_gym_examples package repositories. You can effortlessly incorporate these repositories into your Catkin workspace using the subsequent command:

git clone --recurse-submodules https://github.com/rickstaa/ros-gazebo-gym-ws.git src

The usage of the --recurse-submodules argument ensures the cloning of all submodules. If you have already cloned the repository and neglected the –recurse-submodule argument, you can rectify this by employing the following git command:

git submodule update --init --recursive

Task environment configuration

Configuring the main parameters of a ROS Gazebo Gym task environment is a straightforward process using the gym.make() function. For instance, to modify settings like the maximum number of steps per episode, the reward mechanism, and the control type employed in the PandaReach-v1 task environment, follow this example:

import gym

env = gym.make("ros_gazebo_gym:PandaReach-v1", max_episode_steps=1000, positive_reward=False, control_type="effort")

Additionally, each task environment provides a corresponding yaml configuration file. These files are invaluable for precise fine-tuning and reside in the config folder alongside the respective task environment (e.g. task_envs/panda/config/panda_reach.yaml). When you create a task environment, these configuration files are automatically loaded. The parameters in these configuration files are also directly accessible through the ROS parameter server.

Troubleshooting

This section provides solutions to common issues when using the ROS Gazebo Gym package.

Virtual environments and ROS

When incorporating the ROS Gazebo Gym framework within a virtual environment to maintain the integrity of your system’s Python installation, you can utilize the venv package. Execute the following command to establish a virtual environment within the ros-gazebo-gym folder:

python -m venv ./ros-gazebo-gym --system-site-packages

It’s crucial to include the --system-site-packages flag. This inclusion ensures that the virtual environment gains access to the ROS system packages. Subsequently, you can activate this environment using the command: . ./ros-gazebo-gym/bin/activate.

Attention

You might encounter complications when working with ROS within an anaconda environment (refer to this issue). If you intend to utilize the ROS Gazebo Gym framework within a virtual environment, it’s recommended to opt for the default Python virtual environment package. Alternatively, consider leveraging the RoboStack ros-noetic packages available on conda-forge. For additional insights, consult this enlightening blog post.

Ubuntu 20.04 compatibility issues

When deploying the ros_gazebo_gym package on Ubuntu 20.04, it’s possible to run into complications arising from conflicting versions of the gymnasium and Numpy packages. This conflict is detailed in this issue. Should you encounter this situation, a recommended strategy involves utilizing a virtual environment and manually installing the required Python dependencies from the requirements.txt file using pip.

Execute the following command to install the dependencies:

pip install -r requirements.txt

Note

While it’s not the preferred approach, we also offer custom rosdep rules to ensure proper installation of the required versions of the gymnasium and Numpy packages, even without utilizing a virtual environment. For For a deeper dive into this option, please consult the README.md file in the rosdep folder.