Add new environments

Welcome to the guide on adding new environments to the ros_gazebo_gym project, an inclusive open-source initiative that invites contributions from the community. Prior to making your own contribution, we strongly encourage a thorough review of the contributing guidelines. Below, we provide a concise and structured guide to assist you in introducing a new ROS gazebo gymnasium environment. For any queries that arise, please feel free to open an issue to seek further clarification.

Creating task and robot Environments

General instructions

To develop a new environment, the ros_gazebo_gym package provides two templates to create ROS gymnasium environment wrappers:

  • template_my_robot_env.py: This template constructs the robot environment, responsible for sensor data reading and actuator control.

  • template_my_task_env.py: This template builds the task environment, responsible for initial environment setup, task reward computation, and episode completion checks.

To embark on creating a new environment, establish a new folder for your robot within the ros_gazebo_gym/task_envs directory. Copy the template_my_task_env.py into this folder and rename it accordingly. Then, duplicate the template_my_robot_env.py file into the ros_gazebo_gym/robot_envs directory and provide it a new name. As you proceed, address the TODOS indicated in these templates. It’s crucial that the robot environment inherits from either the RobotGazeboEnv or RobotGazeboGoalEnv classes.

Note

For a dictionary format for observations (as seen in gymnasium Fetch environments), use the RobotGazeboGoalEnv environment.For a simple array observation format, you can inherit from the RobotGazeboEnv class.

A variety of helper classes and methods are accessible to assist in creating your ROS gymnasium environment:

  • ControllersConnection: Manages resetting, switching, and loading ROS controllers.

  • GazeboConnection: Facilitates interaction with the Gazebo simulation, offering access to model and link information, pause/unpause functionality, model spawning, and simulation resetting.

  • LazyImporter: Lazily imports ROS python packages to avoid import errors.

  • ROSLauncher: Launches ROS launch files in a safe subprocess manner.

  • initialize(): Ensures the presence of a running ROS master and Python script-initialized ROS environment.

  • load_ros_params_from_yaml(): Imports ROS parameters from a yaml file.

  • helpers: Contains a variety of helper methods for ROS gazebo gymnasium environments.

Refer to the Python API documentation for a comprehensive overview of all functions and classes within the ros_gazebo_gym package.

Recommendations

Package structure

When constructing your environment, it’s advisable to house ROS launch files within a distinct ROS package. This separation ensures the cleanliness of the ros_gazebo_gym package. Incorporate this dependency into the ros_gazebo_gym/cfg/rosdep.yaml file:

 1# Archive with download information about the ROS dependencies used in the 'ros_gazebo_gym' environments.
 2# NOTE: Here you should add information about ROS dependencies for new 'ros_gazebo_gym' environments.
 3# NOTE: Used in the ROSLauncher class.
 4
 5# == Panda env ==
 6panda_gazebo:                                             # Main environment package dependency.
 7  binary: False                                           # If also available as system binary.
 8  git_url: https://github.com/rickstaa/panda_gazebo.git   # Git repository URL.
 9  git_branch: noetic                                      # Git repository branch.    
10
11  # Additional dependencies.
12  # NOTE: If possible specify in main environment package 'package.xml' file.                       
13  deps:
14    # franka_ros:  # NOTE: Included as an example, redundant with 'panda_gazebo' submodule.
15    #   binary: True
16    #   git_url: https://github.com/frankaemika/franka_ros
17    #   git_branch: develop

After incorporating dependencies, utilize the ROSLauncher to execute your launch files. This class guarantees that all prerequisites for your environment are installed and constructed. For an example, refer to the PandaEnv and the panda_gazebo package.

Environment configuration

It’s recommended to store task environment configuration parameters in a dedicated configuration file located within a config folder in your task environment directory. These parameters can be loaded onto the ROS parameter server using the ros_gazebo_gym.core.helpers.load_ros_params_from_yaml() method. This enables their use in both the Task and Robot classes. An example can be found in the PandaReachEnv environment. The configuration parameters are stored in the panda_reach.yaml file:

 1# This config file contains the configuration values that are used by the Panda reach task
 2panda_reach:
 3  ########################################
 4  # Control settings #####################
 5  ########################################
 6  control:
 7    direct_control: True # Directly control the panda robot by publishing on the controller command topics (FAST). When ``False`` the 'panda_gazebo' control services will be used (SLOWER).
 8    # ee_link: "panda_link8" # Uncomment to change end effector. Defaults: 'panda_hand' if gripper loaded, 'panda_link8' otherwise.
 9    # NOTE: Gripper not loaded due to gravity compensation issues with effort control (see https://github.com/frankaemika/franka_ros/issues/160#issuecomment-992776684).
10    load_gripper: False # Whether you want to load the gripper.
11    # lock_gripper: True # Whether the gripper should be locked (i.e. not move).
12    grasping: False # Whether the gripper should grasp objects. If true a max effort of 10N will be used when not specified otherwise.
13    arm_wait: False # Wait for the arm control to complete before continuing to the next step.
14    hand_wait: False # Wait for the hand control to complete before continuing to the next step.
15    ee_control_coordinates: # Control variables used during ee-control.
16      - "x"
17      - "y"
18      - "z"
19      - "rx"
20      - "ry"
21      - "rz"
22      - "rw"
23    controlled_joints: # Joints that are controlled during joint position/effort control.
24      - "panda_joint1"
25      - "panda_joint2"
26      - "panda_joint3"
27      - "panda_joint4"
28      - "panda_joint5"
29      - "panda_joint6"
30      - "panda_joint7"
31      # NOTE: The current implementation only accepts gripper_width not individual finger joints.
32      # - "gripper_width"
33      # NOTE: When uncommented the robot also tries to control the max gripper effort. Only used when 'gripper_width' is controlled.
34      # - "gripper_max_effort"
35    locked_arm_joints: # Arm joints that are locked during control.
36    #   - "panda_joint1"
37    #   - "panda_joint2"
38    #   - "panda_joint3"
39    #   - "panda_joint4"
40    #   - "panda_joint5"
41    #   - "panda_joint6"
42    #   - "panda_joint7"

These parameters are loaded onto the ROS parameter server within the ros_gazebo_gym.task_envs.panda.panda_reach.PandaReachEnv.__init__() method using the ros_gazebo_gym.task_envs.panda.panda_reach.PandaReachEnv._get_params() method.

Registering the environment

Upon creating your environment, it must be registered within the ros_gazebo_gym package to be discoverable by gymnasium. Achieve this by incorporating information about your environment within the ros_gazebo_gym.task_envs.task_envs_list configuration file:

1# TODO: Update reward thresholds.
2ENVS = {
3    # Panda task envs.
4    "PandaReach-v1": {
5        "entry_point": "ros_gazebo_gym.task_envs.panda.panda_reach:PandaReachEnv",
6        "reward_threshold": -20,
7        "max_steps": 500,
8    },
9    "PandaPickAndPlace-v1": {

Each environment entry should include module, max_steps, and reward_threshold attributes. Following this addition, the ros_gazebo_gym package will facilitate the environment’s registration under the gymnasium namespace. This will enable environment creation using the gym.make() method.