Integrate your own workflow with job automation APIs

OAsis offers job automation APIs that allow you to enqueue and inspect jobs easily. This means you can seamlessly integrate HPC into your workflow engine as a component. For example, you can use it to finetune machine learning models with new data or set up simulation jobs using your own tooling.

This article will use the Molecular Dynamics Simulation case as an example, but the concept should generally apply to all types of workloads.

For the simulation of the ApoA1 protein in a water box and the study of energy changes, we will be using NAMD. NAMD is available in various distributions, with OAsis offering it as both the Lmod system and the container version packaged by NVIDIA NGC. We will use the container version in this article.

Run the simulation using quick job

We can run container-based jobs easily in the web portal. Access Jobs > Run container and fill in the following:

Enqueue the job, and you should see the job output (log file) on the following screen:

Please wait until the job is completed. Once that's done, repeat step 1, but click on the "Export job script" button this time. This will allow you to download a quickjob.sbatch file. This file is a standard SLURM job script that can be enqueued using the sbatch command.

Upload it back to the cluster via the file browser to finish the setup.

Run the job programmatically with the hc CLI client

Ensure you are using the latest hc client before proceeding. This article is tested with hc version 0.0.196.

We are going to use two APIs: enqueue-job and job-status. You may inspect their usage using the --help argument.

Let's use the two sub-commands to enqueue a new job and then inspect it's status.

$ hc enqueue-job --path=/pfss/home/loki/quickjob.sbatch
Submitted batch job 164337

$ hc job-status -j 164337
JobState:completed