.. _virtualgl_sec: ============================= Remote desktop with VirtualGL ============================= .. highlight:: bash For large SMITER studies and Linux cluster access to visualisation nodes from "thin clients" there are several solution availble. All are based on `VirtualGL `_ technology that utilizes hardware OpenGL rendering. Therefore, visualisation nodes need to have `NVidia graphics card `_ (GPU) installed and configured for GPU access by users. Commercial solutions: - https://www.nomachine.com - https://www.nice-software.com/products/dcv - https://www.cendio.com/ Usually, there is some effort needed by the user to achieve VirtualGL destop if not using some of the commercial solutions. To ease this desktop setup there are some custom (open source) solutions for automatic desktop creation: - https://www.massive.org.au/userguide/cluster-instructions/strudel - https://gitlab.version.fz-juelich.de/vis/strudel - `Remote Connection Manager `_ These remote VNC desktop connections are actually doing SSH connections to visualisation node and can be obtained also manually. See NVIDIA white paper on `Remote Visualization on Server-Class Tesla GPUs `_ for insight of the technology. There are possibilites to do such node allocation and use of TurboVNC with VirtualGL in "manual" by firstly allowing access to the node and then running VNC server on it. The following example shows how to setup an interactive session with SLURM batch manager to node with GPU, establishing VNC server and connecting to it with remote client. Allowing remote access to compute nodes ======================================= When user gets allocation to the visualisation compute node with `SLURM `_ workload manager command:: $ salloc --nodes=1 -t 1:00:00 -A m.2017-1-487 $ echo $SLURM_NODELIST t01n3 Administrators can allow login to compute node with ``pam_slurm.so`` setup such as:: $ cat /etc/pam.d/sshd #%PAM-1.0 # auth include system-auth auth optional pam_afs_session.so minimum_uid=500 debug=1 program=/ usr/heimdal/bin/afslog account required pam_nologin.so account required pam_slurm.so account include system-auth password include system-auth session optional pam_keyinit.so force revoke session required pam_limits.so session include system-auth session required pam_env.so conffile=/etc/security/ssh_pam_env.co nf session required pam_loginuid.so session required pam_afs_session.so minimum_uid=500 debug=1 program=/ usr/heimdal/bin/afslog It should be enough to add this line to the sshd pam config (and possibly removing existing restrictive pam modules like ``pam_access`` if they are enabled). It is also strongly suggested looking at ``pam_slurm_adopt`` that puts the ssh-initiated session in the same process containment as the rest of the job. This is especially important if one share nodes with other jobs. Manual setup of the VNC server on the node with VirtualGL ========================================================= On a compute node with VirtualGL installed one needs to create TurboVNC server to which users can then connect through a SSH tunnel as described in the follwing script:: #!/bin/sh set -x XDISPLAY=:0 XVNCDISPLAY=:1 export DISPLAY=$XDISPLAY Xorg $XDISPLAY -nolisten tcp & sleep 5 /opt/TurboVNC/bin/Xvnc -securitytypes None -localhost -nolisten tcp \ -geometry 1920x1080 \ -fp /usr/share/X11/fonts/misc $XVNCDISPLAY & sleep 5 /usr/bin/mwm -display :1 & /usr/bin/xterm -display :1 & /usr/bin/xsetroot -gray -display :1 & cn=$(hostname) cat << __EOF__ Use the following command on your local machine: export VNC_VIA_CMD='/usr/bin/ssh -f -L "\$L":"\$H":"\$R" "${USER}@\$G" "sleep 2 0"' vncviewer -via ${cn} localhost:1 __EOF__ On local linux one then type command:: leon@dell5520a:~/smiter$ export VNC_VIA_CMD='/usr/bin/ssh -f -L "$L":"$H":"$R" "kosl@$G" "sleep 20"' leon@dell5520a:~/smiter$ vncviewer -via t02n47.pdc.kth.se localhost:1 and when TigerVNC or TurboVNC desktop appears in terminal window issues:: [ kosl@t02n47 ] ~ $ cd smiter/salome/ [ kosl@t02n47 ] ~/smiter/salome $ vglrun ./smiter to get the folowing desktop when opening and running one study. .. image:: images/virtualgl.png GPU memory usage ================ Simple cases are very light in terms of GPU memory usage. Inres1 case takes arounf 10MB of GPU memory as shown wtith the following command available with Nvidia cards: .. code-block:: console $ nvidia-smi +-------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================| | 0 19699 G Xorg 41MiB | | 0 23691 G SALOME_Session_Server 10MiB | +-------------------------------------------------------------+ Larger cases such as NF55 can take 1GB of memory. Shadow mesh alone with 2.5 milions triangles the memory usage is: .. code-block:: console +------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |============================================================| | 0 19699 G Xorg 42MiB | | 0 20484 G glxgears 4MiB | | 0 20643 G SALOME_Session_Server 237MiB | +------------------------------------------------------------+ When loading all NF55 meshes in memory it consumes 1GB of GPU memory and aroung 5GB physical (system) and 7GB virtual memory. .. code-block:: console +------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |============================================================| | 0 19699 G Xorg 41MiB | | 0 20484 G glxgears 4MiB | | 0 20643 G SALOME_Session_Server 1031MiB | +------------------------------------------------------------+ Overall, for single user a graphics card with 2GB of memory should be sufficient for normal use. Seting up VirtualGL server ========================== We recommend *lightdm* X11 display manager to be enabled and the rest disabled accordingly with ``systemctl enable/disable .service`` commands. For Example when everything is correct then one can check with:: # systemctl status lightdm.service * lightdm.service - Light Display Manager Loaded: loaded (/usr/lib/systemd/system/lightdm.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2018-09-10 13:52:10 CEST; 3s ago Docs: man:lightdm(1) Main PID: 25106 (lightdm) CGroup: /system.slice/lightdm.service +- /usr/sbin/lightdm +- /usr/bin/X -core -noreset :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt1 -novtswitch -background none and then check that:: # systemctl get-default returns ``graphical.target`` otherwise we need to enable with:: # systemctl set-default graphical.target to be persistent during reboots. If having multiple NVIDIA graphics cards then the example config from ``io-ls-titan.iter.org`` in file:`smiter/salome/examples/virtualgl/xorg.conf` shows how to setup :file:`/etc/X11/xorg.conf`. .. _howto_virtualg_windows: Remote access to compute node with VNC on Microsoft Windows =========================================================== To connect to a remote compute node with Microsoft Windows, the following software has to be installed: * `PuTTY `_ * `TurboVNC `_ To install the software simply go to the site and go to the download site. Then download the correct 32 or 64 bit version and install it. Or click on the following links to the **MSI installer** files. * `PuTTY 32 bit `_ * `PuTTY 64 bit `_ * `TurboVNC 32 bit `_ * `TurboVNC 64 bit `_ Setting up VNC server using PuTTY --------------------------------- First a profile has to be set with the correct settings in PuTTY. With this we will enable a ``SSH`` tunnel to one of the ``hpc-login*.iter.org`` login nodes. Run PuTTY by pressing :kbd:`Win-r` and writing ``putty`` into the dialog. .. image:: images/windowsVNC_1.png :align: center This will open PuTTY. Traverse to :menuselection:`Connection --> SSH --> Tunnels`. .. image:: images/windowsVNC_2.png :align: center Set the :guilabel:`&Source port` to ``5900`` and :guilabel:`Destination` to ``hpc-login8.iter.org:5902``. .. image:: images/windowsVNC_3.png :align: center Click on :guilabel:`A&dd`. This creates a new entry in ``Forwarded ports`` and the entry will be seen as ``L5900 hpc-login8.iter.org:5902``. Now head to the :menuselection:`Session` section, which is located at the top in the left scroll area. Set the ``Host name (or IP address)`` to ``$USER@hpc-login8.iter.org`` where ``$USER`` is the username on the login node. In this case the value for ``$USER`` is ``simicg``. .. image:: images/windowsVNC_4.png :align: center Now save the settings by setting :guilabel:`Sav&ed Sessions` to ``ITER8-VNC`` and then clicking :guilabel:`Save`. The name for the session can generally be anything. This saves the settings for the session for future use. .. image:: images/windowsVNC_5.png :align: center Click on :guilabel:`&Open`, to open the connection. This opens a terminal in which immediatly should ask for your password. Enter the password. .. image:: images/windowsVNC_6.png :align: center Now before we set a tunnel from **hpc-login** to **io-ls-titan** we will go and start the ``VNC server`` on **io-ls-titan**. Write the following command:: $ ssh io-ls-titan Accept the key-print if there is a message about it and enter the password if you are prompted for it. Now we will setup the VNC server by running ``/opt/TurboVNC/bin/vncserver``:: [simicg@io-ls-titan ~]$ /opt/TurboVNC/bin/vncserver You will require a password to access your desktops. Password: When you run the VNC server for the first time, it prompts you to set up a password with which you will access the desktops, created by VNC. Enter a password and do not forget it:: Verify: Would you like to enter a view-only password (y/n)? Write ``n``:: Desktop 'TurboVNC: io-ls-titan.iter.org:2 (simicg)' started on display io-ls-titan.iter.org:2 Creating default startup script /home/ITER/simicg/.vnc/xstartup.turbovnc Starting applications specified in /home/ITER/simicg/.vnc/xstartup.turbovnc Log file is /home/ITER/simicg/.vnc/io-ls-titan.iter.org:2.log This started a VNC server on display ``:2`` and created a configuration directory in ``home`` directory. In this case the display number is ``:2``, so now kill the server with the command:: [simicg@io-ls-titan ~]$ /opt/TurboVNC/bin/vncserver -kill :2 Killing Xvnc process ID 7413 We will now edit the configuration file ``~/.vnc/xstartup.turbovnc`` so that it starts a desktop session when the VNC server starts. Perform the following commands:: $ cd ~/.vnc $ mv xstartup.turbovnc xstartup.turbovnc.orig $ echo "exec env PATH=/opt/trinity/bin:\${PATH} starttde" > xstartup.turbovnc $ chmod 755 xstartup.turbovnc Now restart the VNC server and don't forget the number of the display on which it starts. The output this time will be different:: [simicg@io-ls-titan ~]$ /opt/TurboVNC/bin/vncserver Desktop 'TurboVNC: io-ls-titan.iter.org:2 (simicg)' started on display io-ls-titan.iter.org:2 Starting applications specified in /home/ITER/simicg/.vnc/xstartup.turbovnc Log file is /home/ITER/simicg/.vnc/io-ls-titan.iter.org:2.log Now exit the ``io-ls-titan`` node:: [simicg@io-ls-titan ~]$ exit logout Connection to io-ls-titan closed. .. note:: The display number of VNC server also tells us on which port it will broadcast. In this case, the display number is ``2`` so the port number on ``io-ls-titan`` shall be ``5902``. We will now perform a ``SSH`` forwarding from ``io-ls-titan`` to ``hpc-login8.iter.org``:: [simicg@hpc-login8 ~]$ ssh -g -L 5902:io-ls-titan:5902 io-ls-titan Last login: Mon Jul 2 14:30:36 2018 from hpc-login8.iter.org .. note:: In case the display number of VNC server is not ``2`` and for example is ``10``, the command would be:: [simicg@hpc-login8 ~]$ ssh -g -L 5902:io-ls-titan:5910 io-ls-titan Now the connection is established and we can connect to the ``io-ls-titan`` login node via ``TurboVNC`` viewer in ``Microsoft Windows``. Do **not** close the PuTTY terminal. Running TurboVNC viewer in Microsoft Windows -------------------------------------------- Start TurboVNC viewer by pressing :kbd:`Win` key and writing ``TurboVNC``. Click on ``TurboVNC Viewer``. In the :guilabel:`VNC server` write ``localhost``. .. image:: images/windowsVNC_7.png :align: center A prompt for the ``password`` will show up. Enter the password you set for the ``VNC server``. .. image:: images/windowsVNC_8.png :align: center A session will open up and you are now connected to ``io-ls-titan``. .. image:: images/windowsVNC_9.png :align: center Running SMITER in TurboVNC -------------------------- Open a terminal, by pressing :kbd:`Ctrl-t` in TurboVNC viewer. Write the next commands:: module load smiter smiter .. image:: images/windowsVNC_10.png :align: center Running SMITER with virtualGL in TurboVNC ----------------------------------------- To run ``SMITER`` with ``VirtualGL`` open a terminal with :kbd:`Control-t` in TurboVNC viewer. Write the next commands:: Xorg -nolisten tcp :11 .. note:: The value ``:11`` might be already taken, so if the command fails to run for that value, change the value by adding 1 (``:12``) and run again until it works. Remember the value with which you ran the ``Xorg`` command. Open a new tab in the terminal by pressing :kbd:`Control-Shift-n`. In that terminal write the following command:: $ /opt/VirtualGL/bin/vglrun -d :11.0 /home/ITER/kosl/smiter/salome/smiter .. note:: The value ``-d :11.0`` corresponds to the value with which you ran ``Xorg``. That means if you couldn't run the the previous command with the value ``-d :11.0`` then run it with the value you ran ``Xorg``.