Submitting jobs

To cater to users from different knowledge backgrounds with different preferences, we provide various ways to use the cluster's resources.

SLURM is our job scheduler. Basically, when you need computing resources, you submit a job. The system will then verify your request and your quota, your job will be submitted to the SLURM queue when it is valid. Based on an evaluated priority, SLURM decides when and where your job will be executed.

Besides using the command line interface to submit jobs, we have several quick jobs to get you the resources with a little learning effort. This way suits people that are more comfortable working with GUI.

Jupyter Lab

In just a few clicks, you can launch a Jupyter Lab server of any size and connect to it without further authentication. The Jupyter Lab is managed in our base environment. So you can use it without any preparation. But you will need to create your own Anaconda environment to install the packages you need. Log in to the console and type commands:

module load Anaconda3/2022.05

# for example we create an environment called torch
# install two packages, pip for package management and ipykernel for running our python codes
conda create -y -n torch pip ipykernel

# install pytorch into our environment from the pytorch repo
conda install -y -n torch -c pytorch pytorch

Now we are all set. Login to the web portal, locate the jobs dropdown and click Jupyter Lab. In the GUI launcher window, select the resources you need then click enqueue job. Head to jobs > running jobs to find your job. Click the Jupyter link to open your Jupyter Lab.

Besides Jupyter Lab, you may launch and connect any web-based tools in the same way. See the GUI launcher section for details.

VNC

Some software doesn't provide a web interface but is a desktop application. In this situation, requesting a VNC server comes in handy. Same as Jupyter Lab, the VNC server is running in a node that provides you with the requested compute resources.

Login to the web portal, locate the jobs dropdown and click VNC. In the GUI launcher window, select the resources you need then click enqueue job. The resolution will be automatically set for you, you usually don't have to change it. Head to jobs > running jobs to find your job. Click the VNC link to connect with our web-based VNC client.

We suggest using containers to run your GUI applications so no need to struggle with the UI toolkits. Following is an example of running RStudio with our provided containers. Run in your console to create a shortcut on your VNC desktop.

mkdir -p ./Desktop

echo '
[Desktop Entry]
Version=1.0
Type=Application
Name=RStudio
Comment=
Exec=singularity run --app rstudio /pfss/containers/rstudio.3.4.4.sif
Icon=xfwm4-default
Path=
Terminal=false
StartupNotify=false
' > ./Desktop/RStudio.desktop

chmod +x ./Desktop/RStudio.desktop

Container

The Jupyter Lab and VNC approaches are good for interactive workloads. For non-interactive single-node jobs, you have another handy option for you. You may enqueue a container job with the quick job launcher.

Login to the web portal, locate the jobs dropdown and click Run Container. In the launcher window, select the resources you need, select a built-in or custom container from the picker, and type in the command and a path to store the output. Then click enqueue now to let the job scheduler help. You can then head to jobs > running jobs to check the progress.

When our built-in containers don't fit your need, you may build your own image from scratch or extend our containers. We will cover this later.

GUI launcher

The above three quick jobs are all leveraging the GUI launcher. You can modify them or even create your own quick jobs. They are basically just a .sbatch script, optionally plus some metadata.