Prepare custom software and quick jobs

When the built-in software doesn't fit your needs, feel free to bring your software to the cluster. This article covers how you can do this in Lmod and containers and how to share it with your teammates.

Lmod

First, please study the official Lmod guide about Personal Modulefiles. Then we recommend you place your software and modulefiles in the group scratch file set. Make sure to make all directories and files readable by your group. If you don't want your teammate to modify it, make it writable only by the owner.

Following is an example of compiling git 2.38.1 and adding it as a custom module:

# define where to put our software and modulefiles
MODHOME=/pfss/scratch02/appcara
PKGPATH=$MODHOME/pkg
MODPATH=$MODHOME/modulefiles

# download source code and compile
cd $MODHOME
wget https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.38.1.tar.gz
tar xf git-2.38.1.tar.gz
cd git-2.38.1
./configure --prefix=$PKGPATH/git/2.38.1
make && make install

# setup the module file
mkdir -p $MODPATH/git
cat > $MODPATH/git/2.38.1.lua <<EOF
local home    = "/pfss/scratch02/appcara"
local version = myModuleVersion()
local pkgName = myModuleName()
local pkg     = pathJoin(home, "pkg", pkgName, version, "bin")
prepend_path("PATH", pkg)
EOF

Now everyone who has access to your group scratch directory can use your new module with the following commands.

# use the custom module path
module use /pfss/scratch02/appcara/modulefiles

# check if our git is available
module avail git

# load the module and test
module load git/2.38.1
git --version   # you should see git version 2.38.1

Containers

method 1 - sandbox 
singularity build --sandbox new_alpine/ /singularity/alpine.3.15.3.sif
singularity exec new_alpine vim
singularity exec --writable new_alpine apk update 
singularity exec --writable new_alpine apk add vim
singularity exec new_alpine vim
singularity build new-alpine.sif new_alpine
rm -rf new_alpine
singularity build --sandbox new_alpine/ new-alpine.sif
singularity exec new-alpine.sif vim

method 2 - use Singularity Definition Files
singularity build alpine-with-vim.sif /test/new-alpine.def
singularity exec alpine-with-vim.sif vim

The following example shows how to install the Gym library from OpenAI onto our PyTorch image for reinforcement learning research.

vi gym.def
# BootStrap: localimage
# From: /pfss/containers/pytorch.22.09-py3.sif
#
# %post
#     pip install gym==0.24.1 gym[atari,accept-rom-license]==0.24.1
#     pip install atari-py==0.2.9 pybullet==3.2.5

They are just a .sbatch script, optionally, plus some metadata.