Prerequisites¶

Hardware¶

Overview of an example setup.

1x Microsoft kinect v2.
1x Panda Emika Franka Robot.
A laptop or PC with at least 12gb RAM and a decent Graphics card.

Software¶

ROS¶

The panda-autograsp package has only been tested with ROS Melodic and Kinect. The ROS melodic installation instructions can be found here. The ROS kinetic installation instructions can be found here.

You further also need the following ROS packages:

sudo apt-get install ros-melodic-moveit-ros-move-group ros-melodic-controller-manager* ros-melodic-moveit* ros-melodic-effort-controllers ros-melodic-joint-trajectory-controller ros-melodic-gazebo-ros* ros-melodic-rviz* libboost-filesystem-dev libjsoncpp-dev python-catkin-tools libspatialindex4v5 libspatialindex-dev python-rtree

Warning

As ROS melodic is the recommended version, the instructions below are for ROS Melodic. Change melodic to kinetic in each command if you want to use ROS kinetic.

Python¶

As ROS melodic does not yet fully support python 3, the panda-autograsp package has only been tested with Python 2.7.

Ubuntu¶

The panda-autograsp package has only been tested with ubuntu 18.04.

Libfreenect2 library¶

To use the package together with the Kinect camera, we need to install the libfreenect2 library. The documentation for installing the libfreenect2 can be found in the readme of the repository.

Libfranka library¶

In order to use this package with the panda emika franka robots you need to build the Libfranka library from source. The steps for doing this can be found in the libfranka documentation.

Note

If you want to use the panda-autograsp package with the real robots you also have to install the real-time kernel.

Virtualenv¶

Although the package can be installed on the system python you are advised to use a python environment management system like Virtualenv or conda.

Warning

As ROS doesn’t play nicely with anaconda I wrote a small ROS Conda_wrapper. Unfortunately, I wasn’t yet able to solve all the problems caused case by this CONDA ROS incompatibility. You are therefore currently advised to use a Virtualenv instead.

CUDA & CUDNN (NVIDIA GPUs only)¶

Since Tensorflow needs GPU computing capabilities of your NVIDIA graphics card, the CUDA v10.0 and CDNN v7.6.5 toolkits need to be installed. For installation instructions, you are referred to the CUDA, CUDNN or tensorflow documentation. Additionally, these toolkits are included in the Conda version of the python opencv package. You can install this package by running the conda install -c conda-forge opencv command in your Conda environment.

AMD build of tensorflow (AMD GPUs only)¶

A guide explaining how to use GPU computing with a AMD graphics card be found here.

Package build from source instructions¶

Clone the repository and build the package¶

Clone or download the panda-autograsp catkin package from Github:

bash -c "mkdir -p ~/panda_autograsp_ws \
&& cd ~/panda_autograsp_ws \
&& source /opt/ros/melodic/setup.sh \
&& git clone --recursive https://github.com/rickstaa/panda-autograsp.git src"

Install the ROS system and python dependencies¶

Install the ROS package dependencies using the following command:

rosdep install --from-paths src --ignore-src --rosdistro melodic -y --skip-keys libfranka

Since ROS melodic uses the system python we have to make sure we use the –system-site-packages flag while creating the virtual environment. This makes sure our package can also access the required system python packages. This results in the following command:

virtualenv --system-site-packages ~/venvs/panda_autograsp

Install python package using pip¶

As rosdep does not yet support specifying specific versions for python packages, we need to install some additional packages using the pip install command. To ease this process a setup.py file was created. This file can be invoked using the following commands:

source ~/venvs/panda_autograsp/bin/activate
cd src
export LIBFREENECT2_INSTALL_PREFIX="<PATH_TO_FREENECT2>"
pip install -e .
pip uninstall opencv-python

Note

The opencv-python package, that is installed by one of the python dependencies, has to be removed since it conflicts with required opencv-contrib-python package.

Warning

After 1-01-2020 a lot of packages have dropped python2 support. As a result the above installation method might not work anymore. If this is the case you are advised to install the python requirements using the ./requirements/requirements.txt file.

Build the package¶

The catkin package can be build by executing one of the following commands:

catkin build -j4 -DCMAKE_BUILD_TYPE=Release -DFranka_DIR:PATH=<PATH_TO_LIBFRANKA>/libfranka/build -Dfreenect2_DIR=<PATH_TO_FREENECT2>/freenect2/lib/cmake/freenect2"

Singularity Container installation instructions¶

Nvidia container¶

A ready to run Nividia compatible singularity container is provided. This container can be build using the recipe files found in the panda_autograsp/containers/singularity folder or by pulling directly from the singularity-hub.org container registry.

Note

Due to the fact that I wasn’t able to solve the ros_conda_wrapper problem explained above the container, which uses anaconda is not fully ready. I will update the container with the stable ROS Conda_wrapper when it is stable.

Warning

As the Franka real-time kernel does not yet support NVIDIA drivers (see the Libfranka docs) the NVIDIA container can currently only be used with the simulated robot. An AMD container will be created when singularity starts to support AMD graphics cards. Currently, if you want to use this package on the real robot, you, therefore, have to install it and its dependencies manually. Taking a look at the bash code in the ./containers/singularity/Singularity.ros_melodic-cuda10-bionic might ease this process.

1. Build the container¶

The containers in this repository can be pulled directly from the singularity-hub container registry as follows:

build <CONTAINER_NAME>.simg shub://rickstaa/panda-autograsp:ros-melodic-cuda10-bionic

Go to the panda_autograsp/containers/singularity folder and built the container using the recipe file. This is done by running the following command:

sudo singularity <CONTAINER_NAME>.simg shub://rickstaa/panda-autograsp:ros-melodic-cuda10-bionic

You can also add the --sandbox argument to build the container as a writeable folder.

Warning

You need root access to build from a recipe file.

2. Run the container¶

After te container has been build run it using the singularity run --writable <YOUR_CONTAINER_NAME> command.

3. Clone the repository and build the package¶

As the panda-autograsp package is still private it can not be automatically build during the container generation. You, therefore, have to clone and build the package manually after the docker container is build. This is done by running the following commands:

bash -c "mkdir -p ~/panda_autograsp_ws \
&& cd ~/panda_autograsp_ws \
&& source /opt/ros/melodic/setup.sh \
&& git clone --recursive https://github.com/rickstaa/panda-autograsp.git src \
&& catkin build -j4 -DCMAKE_BUILD_TYPE=Release -DFranka_DIR:PATH=/libfranka/build -Dfreenect2_DIR=/opt/freenect2/lib/cmake/freenect2"

Warning

As all of the system dependencies for the panda-autograsp package have already been installed during the container creation, contrary to normal build instructions, for the singularity container, you don’t need to run rosdep install command. If you want to install new system dependencies or run the rosdep install command you have to make sure you start the container as the root user. This is necessary since, in a singularity container, you are the same user inside and outside the container. When developing inside the singularity container, you are therefore advised to place the panda-autograsp workspace on a path which can be both accessed by you and the root user (/opt/ or the container main path / for example).

4. Add additional permissions¶

If you did build the singularity container as a writeable folder, you could give your user write and read access from outside the singularity container by:

Changing the group owner to your user group.

sudo chgrp -R <YOUR_USER_NAME> ./<YOUR_CONTAINER_NAME>

Giving your user group read and write access to the <YOUR_CONTAINER_NAME folder.

sudo chmod -R g+rwx ./<YOUR_CONTAINER_NAME>

AMD compatible container¶

Currently, singularity does not yet support AMD graphics. This option is planned to be included in the next release see this announcement.

Docker container installation instructions¶

We do not yet provide a docker container for this package.