Nimbulus Compute Documentation

Compute nodes are 2x Xeon E5-2698 v3 (32 cores) with 128GB of ram each. 1 Nvidia P4 gpu is available per node (8GB of vram per node).

Storage is provided locally (use your home directory) or on NFS mounts at /nfs and /scratchfs.

• We used to have a nice wired 10gbe connection between the nodes and the headnode. Now we are forced to use Wifi to connect the nodes, so load all your data locally (preferably into /tmp, which is purged periodically), and then export completed results to /nfs
• Use the local jupyter kernel whenever possible

• Use computecanada's precompiled stdenv/avx2 packages via cvmfs to take fullest advantage of the tuned binaries for cp2k. The cp2k packaged by ubuntu default repositories was not compiled to take advantage of the hardware that the xeon processors provide
• When using gpu compute, cast everything to fp32. The tesla p4's performance on fp16 and fp64 are gimped severely.