Avalon Servers
**************

The Avalons are our 4 servers that we use to run many processes in parallel. Have a look
at `Nils' talk <https://sharepoint.uni-goettingen.de/projects/akaj/praesi_thesis/_layouts/15/WopiFrame.aspx?sourcedoc=/projects/akaj/praesi_thesis/Prsentationen/method_seminar_WiSe19_20/avalon_talk.pdf&action=default>`_ for the basics.

Configure SSH Hosts and use SSH Keys
====================================

This makes the whole usage of the Avalons (or any other remote Unix computer)
much more comfortable. With SSH keys, you don't need to type your password anymore,
when you want to log in or copy to/from a remote computer. (It is also considered
much safer to use this for log in whenever you can, best practice would be to
allow log in to avalons only via this, but we also allow log in with password for now.)

If you don't have a SSH key yet (check if the folder ``~/.ssh`` exists), create one with:

.. code::

   ssh-keygen

It should be fine to use the recommended settings. This creates the folder ``~/.ssh``.
There should be a public (``id_rsa.pub``) and a private key file (``id_rsa``) in that folder.
If you want to use SSH in other situations (e.g. for gitlab), make sure you **never copy the private key**
by accident. Never give the private key to anyone.
In my folder there are some additional files:

.. code::

   /home/ilyas/.ssh
   ├── config
   ├── id_rsa
   ├── id_rsa.pub
   └── known_hosts

Create the file ``~/.ssh/config`` and open it in an editor. There you can define what the IP addresses
of hosts are, with which username you want to log in, etc.. For the
avalons, put the following into the file:

.. code::

   Host avalon
        HostName XXX.XXX.XXX.XXX # ask colleagues for IP
        User ilyas
        Port XXXX # ask colleagues for port
   
   Host avalon01
        User ilyas
        ProxyJump avalon
   
   Host avalon02
        User ilyas
        ProxyJump avalon
   
   Host avalon03
        User ilyas
        ProxyJump avalon
   
   Host avalon04
        User ilyas
        ProxyJump avalon

Replace my username ``ilyas`` with your username(s) on the avalons. The first
entry ``avalon`` is actually the head node, not one of the actual computing servers, I hope
that is not too confusing ...

Now in a first step, try to log in to the head node, by entering:

.. code::

   ssh avalon

This should give you some fingerprint that you can accept. After that it will ask you for your
password. Make sure it works and you are logged in. Then log off again by pressing ``ctrl`` + ``d``.
Now copy your ssh key to the head node with:

.. code::

   ssh-copy-id avalon

For that you have to enter your password as well. From now on, you should be able to
log in without a password, but rather with your SSH key. Try to log in again with

.. code::

   ssh avalon

Once that works, you can to the same with one of the avalons. E.g. if you want
to use avalon04:

.. code::

   # make sure your password and account are set up by logging in once:
   ssh avalon04
   # log off again, then enter:
   ssh-copy-id avalon04

That's it!


``rsync`` instead of ``scp``
============================

In many situations, ``rsync`` has some advantages over ``scp``.
It is quite easy to use ``rsync`` to copy only files that are newer
in the source directory than in the destination directory using the
option ``-u``. For that to work more neatly when copying files
back and forth between different machines/drives multiple times,
I recommend to also use the option ``-t`` to keep the time stamps
when you copy files.

I simply use ``rsync`` with the following options most of the time:

.. code::

   rsync -ruvt src dst

The options ``-ruvt`` include the two options described above. In addition,
``-r`` makes it recursive (includes sub-folders), ``-v`` (verbose) leads
to some more feedback printed to the command line.

Example
-------

TODO

Let's say you want to copy your local folder ...