Skip to content

Setup of a Simulation Server

Jump to bottom
Benjamin Délèze edited this page Jun 3, 2022 · 3 revisions

Once a session server is installed, you can proceed with the installation of one or several simulation servers. The first simulation server may run on the same machine as the session server. Others should run on different machines. There should be only one simulation server per machine. Simulation servers can launch several instances of Webots on their machine.

Hardware Requirements

A simulation server should run on any machine with a NVIDIA graphics card. A NVIDIA graphics card is strongly recommended as we found no solution to run 3D hardware accelerated graphics in a Docker with a AMD graphics card. We recommend to use a machine with a powerful multi-core CPU and powerful graphics card. This will allow you to run a large number of Webots simulations in parallel on the same machine. For example, the following systems are a very good configurations:

  • AMD Ryzen 9 5950X & GeForce RTX 3080
  • AMD Ryzen 7 5800X & GeForce GTX 1660 SUPER

It is recommended to configure the BIOS of the computer for automatic reboot after power outage.

Network Requirements

The simulation machines don't need to have their own fully qualified domain name. However, all these machines should be installed on the same local network as the session server. The machine running the session server should be able to access any port of the simulation machines, which is usually the case on local networks.

Software Requirements

If you are installing the simulation server on the same machine as the session server, you can skip steps 1 to 4.

  1. Install Ubuntu 20.04:
    • Choose the desktop version of Ubuntu 20.04. Simulation servers needs a X display and 3D OpenGL hardware acceleration which is easier to setup from a desktop version.
    • create a user account named cyberbotics (or anything else).
  2. Makes that this user account has the auto-login feature enabled, so that when you boot the machine it get automatically logged in.
  3. Configure the unattended upgrades to reboot after security updates.
  4. For your convenience, it is recommended to install ssh: apt install ssh, so that you can login remotely, possibly with display export, e.g., ssh -X.
  5. Install Python 3: sudo apt-get install python3-pip python-is-python3.
  6. Install Python dependencies: pip install pynvml requests psutil tornado.
  7. Install git and subversion: sudo apt-get install git subversion. They are used by the simulation server to checkout the code of the projects.
  8. Install Webots from the source in the ~/webots folder:
    • After performing the git clone, switch to the develop branch: git checkout develop.
    • Don't install any optional dependency unless you need them.
  9. Install docker: sudo apt install docker.io and follow the post-installation instructions: sudo usermod -aG docker $USER and newgrp docker. You will also have to install the NVIDIA docker drivers as documented here. Docker is used by Webots to execute the robot controllers in a security sandbox, so that you can safely execute controller programs (in any programming language, even binaries) coming from the outside world.
  10. Install docker-compose: pip install docker-compose
  11. Optional: make the NVIDIA accelerated X server work also headless (with no screen connected):
    • sudo nvidia-xconfig --allow-empty-initial-configuration

Setup

  1. Install the docker images of the Webots versions that you want to support.

    docker pull cyberbotics/webots:R2022a-ubuntu20.04
docker pull cyberbotics/webots:R2022b-ubuntu20.04

  2. Install the simulation server:

    mkdir ~/simulation_server
cd ~/simulation_server
ln -s /home/cyberbotics/webots/resources/web/server/simulation_server.py
ln -s /home/cyberbotics/webots/resources/web/server/async_process.py

  3. Configure the simulation server: edit ~/simulation_server/simulation.json so that it contains the following (to be adapted to your local setup):

    {
  "webotsHome": "/home/cyberbotics/webots",
  "port": 2000,
  "fullyQualifiedDomainName": "cyberbotics1.epfl.ch",
  "portRewrite": true,
  "logDir": "log/",
  "debug": true
}

  4. Configure Apache 2 on the session server machine to redirect traffic on the simulation machine:

    • If you are running the simulation server on the same machine as the session server, you can skip this step.
    • Otherwise edit /etc/apache2/site-available/000-default-le-ssl.conf and modify the rewrite rules to direct the traffic to the various machines (session server and simulation servers):
    RewriteRule ^/1999/(.*)$ "ws://localhost:$1/$2" [P,L]            # session server
RewriteRule ^/2(\d{3})/(.*)$ "ws://<IP address 2>:$1/$2" [P,L]   # simulation server server with ports in the range 2000-2999
RewriteRule ^/3(\d{3})/(.*)$ "ws://<IP address 3>:$1/$2" [P,L]   # other simulation server server with ports in the range 3000-3999
⋮

RewriteRule ^/1999/(.*)$ "http://localhost:$1/$2" [P,L]        # session server
RewriteRule ^/2(\d{3})/(.*)$ "http://<IP address 2>:$1/$2" [P,L] # simulation server with ports in the range 2000-2999
RewriteRule ^/3(\d{3})/(.*)$ "http://<IP address 3>:$1/$2" [P,L] # other simulation server with ports in the range 3000-3999
⋮

  5. Configure the session server to use this simulation server: edit ~/session_server/session.json to add the simulation server in the simulationServers section:

    ⋮
"simulationServers": [
  "cyberbotics1.epfl.ch/2000"
]
⋮

  6. Setup the automatic launch of the simulation server on reboot.

    cd ~/.config
mkdir -p autostart
cd autostart
echo "[Desktop Entry]" > simulation_server.desktop
echo "Name=simulation_server" >> simulation_server.desktop
echo "Exec=python /home/cyberbotics/simulation_server/simulation_server.py /home/cyberbotics/simulation_server/simulation.json" >> simulation_server.desktop
echo "Type=Application" >> simulation_server.desktop
echo "X-GNOME-Autostart-enabled=true" >> simulation_server.desktop

Tips and troubleshooting

Decrease the graphical quality of Webots

Even if Webots is launched in no-rendering mode, the graphical scene will be computed. It is necessary for the camera to be able to see the scene for example.

However, the graphical quality of the Webots instance that run in the docker will not impact the graphical quality of the simulation displayed in the browser, apart from two exceptions: if you have a camera or if you use local textures (if in your protos/worlds you have images referenced as textures/my_images.jpg and not https://the_address_of_my_picture.jpg).

So, it can be a good idea to decrease the graphical settings to increase the number of instances of Webots that could run in parallel. By changing the graphical settings (that are present in the Preferences/OpenGL tab of Webots) from maximum to minimum, it is possible to multiply the number of parallel instance by 3 or 4 if your GPU is the bottleneck.

To do that, you have to modify the configuration file of Webots (in /$Home/.config/Cyberbotics/Webots-R202??.conf), then create a new docker image. You can use the following Dockerfile:

FROM my_name/my_webots_repo:previous_webots_tag
COPY /path_to_the_configuation_file/Webots-R202??.conf /root/.config/Cyberbotics/Webots-R202??.conf

Put the assets in the docker

Webots (in the docker) will have to load the world first. To reduce this loading time, it is possible to put the assets directly in the docker such that Webots will not need to download them from the web.

You can achieve that with the following Dockerfile:

FROM my_name/my_webots_repo:previous_webots_tag
COPY path_to_your_assets_folder/assets /root/.cache/Cyberbotics/Webots/assets

For more informations about assets, see https://cyberbotics.com/doc/guide/installation-procedure?version=develop&tab-os=linux#asset-cache-download

Clone this wiki locally