Getting started on clusters: Difference between revisions

From Earlham CS Department
Jump to navigation Jump to search
← Older edit
Craigje (talk | contribs)
administrator, Bureaucrats, Administrators
1,085 edits
No edit summary
Pelibby16 (talk | contribs)
administrator, Bureaucrats, Administrators
175 edits
No edit summary
 
(52 intermediate revisions by 6 users not shown)
Line 1: Line 1:
This document presumes zero prior knowledge of cluster computing. If instead you're an intermediate user (e.g. you have an account and have run a few jobs before but need a reminder) the table of contents is your friend.
This document presumes zero prior knowledge of cluster computing. If instead you're an intermediate user (e.g. you have an account and have run a few jobs before but need a reminder) the table of contents is your friend.


This document gives you all the information you need to choose a system, log in to a cluster/phat node, write a script, submit it via qsub to the scheduler, and find the output. As such, these notes cover hardware and software. (If you're a sysadmin, you may be interested in [[Sysadmin:Services:ClusterOverview |this page]] instead.)
This document gives you all the information you need to choose a system, log in to a cluster/phat node, write a script, submit it via sbatch to the scheduler, and find the output. As such, these notes cover hardware and software. (If you're a sysadmin, you may be interested in [[Sysadmin:Services:ClusterOverview |this page]] instead.)
 
Before you get started, make sure you have a CS account. You can email <code>admin@cs.earlham.edu</code> or a current CS faculty member to get started. Your user account will grant access to all the servers below, and you will have a home directory at <code>~username</code> that you can access when you connect to any of them.
 
# First, read about what machines you can pick from. Take a look at [[#Cluster systems to choose from]], and see which machines fit your needs the best.
# Open and terminal and connect to Hopper: <code>ssh username@hopper.cluster.earlham.edu</code>
# Once you are connected to Hopper, you can SSH to the machine you need: <code>ssh username@hamilton.cluster.earlham.edu</code> (In this example, we connect to the Hamilton cluster).
# Now you are ready to start using Slurm on that cluster. Check out [[#Using Slurm]] for more examples and information on how to use the scheduler on that machine.
 


__TOC__
__TOC__


= Prerequisites =  
= Using Slurm =
 
Slurm is our batch scheduler. It lets us run scripts in the background so that you don't have to stay logged and monitor them manually. It also allows us to automatically distribute users/jobs across a set of machines so that we don't all get crowded into one place, helping everyone get to the resources that they need.
 
There are two main ways to use slurm:
# You can submit a job in a script, to run automatically in the background: <code>$ sbatch my_good_script.sbatch</code>
# You can start an interactive job through slurm and maintain manual control: <code>srun -n 1 --pty bash -i</code>


# Get a cluster account. You can email admin at cs dot earlham dot edu or a current CS faculty member to get started. Your user account will grant access to all the servers below, and you will have a home directory at <code>~username</code> that you can access when you connect to any of them.
== Common Directives ==
# Connect through a terminal via ssh to <code>username@hopper.cluster.earlham.edu</code>. If you intend to work with these machines a lot, you should also configure your [[How To Set Up SSH Keys | ssh keys]].
Include these at the top of your sbatch file to configure the scheduler to your needs.


= Cluster systems to choose from =  
* <code>#SBATCH --job-name=myjob</code>: Name your job something specific (shows up in <code>squeue</code>
* <code>#SBATCH --output=file.out</code>: Sets a file to store output from your script in.
* <code>#SBATCH --error=file.err</code>: Sets a file to store errors from your script in.
* <code>#SBATCH --time=HH:MM:SS</code>: Sets a maximum time for your job.
* <code>#SBATCH --nodes=1</code>: Sets the number of nodes to use for the job.
* <code>#SBATCH --cpus-per-task=10</code>: Sets the number of CPUs to use for the job.
* <code>#SBATCH --mem=10G</code>: Sets the amount of memory to use for the job.
* <code>#SBATCH --mail-user=myemail@earlham.edu</code>: Sets the email to send notifications to when the job status changes.
 
== Example sbatch Files ==
=== Example One ===
This job will do the following:
# Print "hellow world!" to the log file.
# end.
 
A few things to notice:
* This is about as simple a job as possible.
* The job is only using one node, and one CPU on that node. Plenty of room for other users to run their jobs in parallel.


The cluster dot earlham dot edu domain consists of clusters (a collection of physical servers linked through a switch to perform high-performance computing tasks with distributed memory) and jumbo servers (nee "phat nodes"; a system comprising one physical server with a high ratio of disk+RAM to CPU, good for jobs demanding shared memory).
<pre>
#!/bin/sh
#SBATCH --job-name hello-world
#SBATCH --nodes=1
#SBATCH -c 1
echo "hello world!"
</pre>


Our current machines are:
=== Example Two ===
This job will do the following:
# Echo some information about the current node (printing it to the output file).
# Print the current machine hostname (to the output file, again).
# sleep for 10 seconds (wait).
# Print the current working directory (to the output file, still).
# end.


* whedon: newest cluster; 8 compute nodes; Torque-only pending an OS upgrade
A few things to notice:
* layout: cluster; 4 compute nodes, pre-whedon, features NVIDIA GPGPU's and multiple CUDA options
* The job is only using one node, and one CPU on that node. Plenty of room for other users to run their jobs in parallel.
* lovelace: newest jumbo server
* The job will notify the user <code>excellent_email_user@earlham.edu</code> via email when it starts, ends, or fails.
* pollock: jumbo server, older than lovelace but well-tested and featuring the most available disk space
* The job has a max runtime of 20 seconds, so if something goes wrong and it doesn't end sooner, the scheduler will stop it.


To get to, e.g., whedon, from hopper, run <code>ssh whedon</code>.
<pre>
#!/bin/sh
#SBATCH --time=20
#SBATCH --job-name hello-world-two
#SBATCH --nodes=1
#SBATCH -c 1 # ask for one core
#SBATCH --mail-type=BEGIN,END,FAIL
#SBATCH --mail-user=excellent_email_user@earlham.edu


If you're still not sure, [[Choosing a computing resource|click here for more detailed notes]].
echo "queue/partition is `echo $SLURM_JOB_PARTITION`"
echo "running on `echo $SLURM_JOB_NODELIST`"
echo "work directory is `echo $SLURM_SUBMIT_DIR`"


= Cluster software bundle =
/bin/hostname
sleep 10          # Replace this sleep command with your command line.
/bin/pwd


The cluster dot earlham dot edu servers all run a supported CentOS version.
</pre>


All these servers (unless otherwise noted) also feature the following software:
=== Example Three ===
This job will do the following:
# Load the <code>python/3.12</code> module environment on the machine it's running on.
# Run the <code>main.py</code> script with the time command (prints information on runtime after the command finishes).
# end.


* Slurm (scheduler): submit a job with <code>sbatch jobname.sbatch</code>, delete it with <code>scancel jobID</code>. Running a job has its own doc section below.
A few things to notice:
* Environment modules: run <code>module avail</code> to see available software modules and <code>module load modulename</code> to load one; you may load modules in bash scripts and qsub jobs as well.
* The job is only using one node, and 30 CPUs on that node.
* The job will notify the user <code>excellent_email_user@earlham.edu</code> via email when it starts, ends, or fails.
* The job has a max runtime of 72 hours, so if something goes wrong and it doesn't end sooner, the scheduler will stop it.


The default shell on all these servers is bash.
<pre>
#!/bin/bash
#SBATCH --job-name=MNISTDCGAN # Job name
#SBATCH --output=output_%j.txt # Standard output file with job id
#SBATCH --error=error_%j.txt # Standard error file with job id
#SBATCH --time=72:00:00 # Maximum run time
#SBATCH --nodes=1 # Use one node
#SBATCH --cpus-per-task=30 # Request 30 CPU cores (or more, as available)
#SBATCH --mem=16G # Total memory
#SBATCH --mail-type=BEGIN,END,FAIL # Email notifications
#SBATCH --mail-user=username@earlham.edu # Your email address


The default Python version on all these servers is Python 2.x, but all have at least one Python 3 module with a collection of widely-used scientific computing libraries.
module load python/3.12 # replace with any python version you want to use
time python main.py # replace with your python file name
</pre>


= Using Slurm =


Slurm is our batch scheduler.
After submitting a job slurm captures anything written to stdout and stderr by the programs and when the job completes puts it in a file called slurm-nnn.out (where nnn is the job number) in the directory where you ran sbatch. Use more to view it when you are looking for error messages, output file locations, etc. If you are used to using <code>qpeek</code>, you can instead just run <code>tail -f jobXYZ.out</code> or <code>tail -f jobXYZ.err</code>.


To submit a job to PBS, you'll need to write a shell script wrapper around it and submit it through qsub on your system of choice. For example (change the specific options):
There's some more CPU management information [https://slurm.schedmd.com/cpu_management.html here].


== Useful Slurm Commands ==
Slurm has some other useful commands that you can use to interact with or view jobs that are running.


{| class="wikitable"
{| class="wikitable"
|+ Commands
|+ Commands
|-
|-
! Torque
! Slurm
! Slurm
! Description
! Description
|-
|-
| <code>qsub</code>
| <code>sbatch</code>
| <code>sbatch</code>
| run/submit a batch job
| run/submit a batch job (.sbatch file, see above)
|-
|-
| <code>qstat</code>
| <code>squeue</code>
| <code>squeue</code>
| show jobs currently in the queue
| show jobs currently in the queue
|-
|-
| <code>qdel</code>
| <code>scancel</code>
| <code>scancel</code>
| cancel a job
| cancel a job by its ID
|-
|-
| <code>pbsnodes -a</code>
| <code>scontrol show nodes</code>
| <code>scontrol show nodes</code>
| show nodes in the cluster
| show nodes in the cluster
Line 72: Line 140:
|+ Environment Variables
|+ Environment Variables
|-
|-
! Torque
! Slurm
! Slurm
! Description
! Description
|-
|-
| <code>$PBS_QUEUE</code>
| <code>$SLURM_JOB_PARTITION</code>
| <code>$SLURM_JOB_PARTITION</code>
| the queue/partition you are in
| the queue/partition you are in
|-
|-
| <code>cat $PBS_NODEFILE</code>
| <code>$SLURM_JOB_NODELIST</code>
| <code>$SLURM_JOB_NODELIST</code>
| there's no equivalent of the nodes file but there is an environment variable that stores that information
| there's no equivalent of the nodes file but there is an environment variable that stores that information
|-
|-
| <code>$PBS_O_WORKDIR</code>
| <code>$SLURM_SUBMIT_DIR</code>
| <code>$SLURM_SUBMIT_DIR</code>
| working directory from which the command was run
| working directory from which the command was run
|}
|}


Example script
= Cluster systems to choose from =
<pre>
 
#!/usr/bin/bash
The <code>cluster.earlham.edu</code> domain consists of clusters (a collection of physical servers linked through a switch to perform high-performance computing tasks with distributed memory) and jumbo servers (nee "phat nodes"; a system comprising one physical server with a high ratio of disk+RAM to CPU, good for jobs demanding shared memory).
 
== Hamilton ==
<code>hamilton.cluster.earlham.edu</code>


#SBATCH --job-name hello-world
(Built in 2022)
#SBATCH --nodes=5
Designed for high CPU/RAM jobs. Used often for photogrammetry work via WebODM and CPU-based AI/ML training.
#SBATCH --mail-type=BEGIN,END,FAIL
#SBATCH --mail-user=excellent_email_user@earlham.edu


echo "queue is `echo $SLURM_JOB_PARTITION`"
=== Services running on this machine ===
echo "running on `echo $SLURM_JOB_NODELIST`"
* [//webodm.cluster.earlham.edu WebODM]
echo "work directory is `echo $SLURM_SUBMIT_DIR`"


srun -l echo "hello world!"
=== Nodes and Hardware ===
</pre>
{| class="wikitable"
|-
! Machine Name !! Type !! CPU !! RAM !! GPU
|-
| h0 || Head Node || AMD EPYC 24-core || 128GB || None
|-
| h1 || Compute Node || AMD EPYC 24-core || 256GB || None
|-
| h2 || Compute Node || AMD EPYC 24-core || 256GB || None
|-
| h3 || Compute Node || AMD EPYC 24-core || 256GB || None
|-
| h4 || Compute Node || AMD EPYC 24-core || 256GB || None
|-
| h5 || Compute Node || AMD EPYC 24-core || 256GB || None
|}


= OLD: Using qsub =
== Faraday ==
<code>faraday.cluster.earlham.edu</code>


Torque is our scheduler. It is gradually being phased out in favor of Slurm but remains installed for now. The command to submit something to be run on the scheduler is qsub. Knowing how to use qsub is the key to using our high-performance computing systems effectively.
(Built in 2022)
Designed for GPU jobs. Used for research, courses, and projects requiring GPU access, computational biophysics simulations, and notebook services.


== Submit job using a .qsub file (recommended) ==
=== Services running on this machine ===
* [//jupyter.cluster.earlham.edu Jupyterhub]


To submit a job to PBS, you'll need to write a shell script wrapper around it and submit it through qsub on your system of choice. For example (change the specific options):
=== Nodes and Hardware ===
{| class="wikitable"
|-
! Machine Name !! Type !! CPU !! RAM !! GPU
|-
| f0 || Head Node || AMD EPYC 16-core || 192GB || Nvidia RTX A5000 24GB
|-
| f1 || Compute Node || AMD EPYC 16-core || 192GB || Nvidia RTX A5000 24GB
|-
| f2 || Compute Node || AMD EPYC 16-core || 192GB || Nvidia RTX A5000 24GB
|-
| f3 || Compute Node || AMD EPYC 16-core || 192GB || Nvidia RTX A5000 24GB
|-
| f4 || Compute Node || AMD EPYC 16-core || 192GB || Nvidia RTX A5000 24GB
|-
| f5 || Compute Node || AMD EPYC 16-core || 192GB || Nvidia RTX A5000 24GB
|}


<pre>
== Whedon ==
#PBS -N out
<code>whedon.cluster.earlham.edu</code>
#PBS -q batch
#PBS -l nodes=1:ppn=1
#PBS -m abe
#PBS -M username@example.com
#PBS -o /path/to/my/stdout
#PBS -e /path/to/my/stderr
</pre>


In this example:
(Built in 2015)
* -N gives the job name
Designed for high CPU/RAM jobs that require long-term processing. Also used for computational Chemistry simulations with WebMO.
* -q tells Torque which queue to submit the job to
* -l tells Torque how many nodes and how many processors-per-node to request
** on a phat node, nodes=1; on a cluster, nodes=N where N>1 (most often)
* -m abe tells Torque what mail options to use. abe enables all mail information.
* -M tells Torque the email address
* -o tells Torque where to copy STDOUT to
* -e tells Torque where to copy STDERR to


It's sometimes useful to include the following snippet as well, so that your output files remind you of some other system details in the days or weeks after your job terminates:
=== Services running on this machine ===
* [//webmo.cluster.earlham.edu WebMO]


<pre>
=== Nodes and Hardware ===
echo "running on `cat $PBS_NODEFILE`"
{| class="wikitable"
echo "hostname is `hostname`"
|-
echo "on launch cwd is `pwd`"
! Machine Name !! Type !! CPU !! RAM !! GPU
echo "PBS_O_WORKDIR is `echo $PBS_O_WORKDIR`"
|-
| w0 || Head Node || 2x Intel Xeon 8-core || 256GB || None
|-
| w1 || Compute Node || 2x Intel Xeon 8-core || 256GB || None
|-
| w2 || Compute Node || 2x Intel Xeon 8-core || 256GB || None
|-
| w3 || Compute Node || 2x Intel Xeon 8-core || 256GB || None
|-
| w4 || Compute Node || 2x Intel Xeon 8-core || 256GB || None
|-
| w5 || Compute Node || 2x Intel Xeon 8-core || 256GB || None
|-
| w6 || Compute Node || 2x Intel Xeon 8-core || 256GB || None
|-
| w7 || Compute Node || 2x Intel Xeon 8-core || 256GB || None
|}


cd $PBS_O_WORKDIR
== Lovelace ==
</pre>
<code>lovelace.cluster.earlham.edu</code>


Using a qsub file is the recommended approach because it is easy to gather your files and share them with others if you have an issue.
Designed for particularly high RAM jobs. Used often for large scale biology projects, such as alignment and sequence analysis. This machine also features a particularly large amount of disk space for such projects.


== Submit job via command line interface ==
=== Services running on this machine ===
* [//rstudio.cluster.earlham.edu RStudio]


You can also run a bash script through qsub via CLI. Here's an example script:
=== Nodes and Hardware ===
{| class="wikitable"
|-
! Machine Name !! Type !! CPU !! RAM !! GPU
|-
| lovelace || Phat Node || 2x Intel Xeon 16-core || 1000GB || None
|}


<tt>
= Cluster software bundle =
#!/bin/sh
#PBS -N primes_block
#PBS -o /path/to/out.txt
#PBS -e /path/to/err.txt
#PBS -q batch
#PBS -m abe
echo "Hello world!" <br>
exit $! <br>
</tt>


* To submit that job, you might run something like this on b0:
The cluster dot earlham dot edu servers all run a supported Debian GNU/Linux 12.
<pre>$ qsub -l nodes=8,cput=0:30:0 ./hello.sh</pre>
This will request 8 nodes for a total of 30 minutes CPU time (granted, this is an absurdly long time to request for "hello world").


== Using qsub interactively ==
All these servers (unless otherwise noted) also feature the following software:


Often it is useful to test commands, scripts, etc. before submitting them to the queue. To do that you can launch qsub interactively and the scheduler will assign resources for use from a shell, e.g. this command would assign two nodes with two processors on each node: <pre>$ qsub -I -l nodes=2:ppn=2 </pre>
* Slurm (scheduler): submit a job with <code>sbatch jobname.sbatch</code>, delete it with <code>scancel jobID</code>. Running a job has its own doc section below.
* Environment modules: run <code>module avail</code> to see available software modules and <code>module load modulename</code> to load one; you may load modules in bash scripts and sbatch jobs as well.


After launching you will be at a new shell prompt. The command: <pre>$ cat $PBS_NODEFILE</pre> will show you which node(s) were assigned to your interactive job.
The default shell on all these servers is bash.


== Related commands ==
The default Python version on all these servers is Python 2.x, but all have at least one Python 3 module with a collection of widely-used scientific computing libraries.


Including qsub for search convenience, here are some common Torque commands you'll probably use (in descending order of probable frequency):
= About qsub =


* qsub: Submit a job to a queue.
Before Slurm we used Torque and its associated software, including qsub. This is now deprecated and should not be used on the Earlham CS cluster systems.
* qstat: Show a list of running, queued, or recently ended jobs.
* qdel: Delete a job you've submitted.
* qhold: Hold a job from execution.


Each command has a man page.
Tested and working 2022

Latest revision as of 20:12, 2 April 2026

This document presumes zero prior knowledge of cluster computing. If instead you're an intermediate user (e.g. you have an account and have run a few jobs before but need a reminder) the table of contents is your friend.

This document gives you all the information you need to choose a system, log in to a cluster/phat node, write a script, submit it via sbatch to the scheduler, and find the output. As such, these notes cover hardware and software. (If you're a sysadmin, you may be interested in this page instead.)

Before you get started, make sure you have a CS account. You can email admin@cs.earlham.edu or a current CS faculty member to get started. Your user account will grant access to all the servers below, and you will have a home directory at ~username that you can access when you connect to any of them.

  1. First, read about what machines you can pick from. Take a look at #Cluster systems to choose from, and see which machines fit your needs the best.
  2. Open and terminal and connect to Hopper: ssh username@hopper.cluster.earlham.edu
  3. Once you are connected to Hopper, you can SSH to the machine you need: ssh username@hamilton.cluster.earlham.edu (In this example, we connect to the Hamilton cluster).
  4. Now you are ready to start using Slurm on that cluster. Check out #Using Slurm for more examples and information on how to use the scheduler on that machine.


Using Slurm

Slurm is our batch scheduler. It lets us run scripts in the background so that you don't have to stay logged and monitor them manually. It also allows us to automatically distribute users/jobs across a set of machines so that we don't all get crowded into one place, helping everyone get to the resources that they need.

There are two main ways to use slurm:

  1. You can submit a job in a script, to run automatically in the background: $ sbatch my_good_script.sbatch
  2. You can start an interactive job through slurm and maintain manual control: srun -n 1 --pty bash -i

Common Directives

Include these at the top of your sbatch file to configure the scheduler to your needs.

  • #SBATCH --job-name=myjob: Name your job something specific (shows up in squeue
  • #SBATCH --output=file.out: Sets a file to store output from your script in.
  • #SBATCH --error=file.err: Sets a file to store errors from your script in.
  • #SBATCH --time=HH:MM:SS: Sets a maximum time for your job.
  • #SBATCH --nodes=1: Sets the number of nodes to use for the job.
  • #SBATCH --cpus-per-task=10: Sets the number of CPUs to use for the job.
  • #SBATCH --mem=10G: Sets the amount of memory to use for the job.
  • #SBATCH --mail-user=myemail@earlham.edu: Sets the email to send notifications to when the job status changes.

Example sbatch Files

Example One

This job will do the following:

  1. Print "hellow world!" to the log file.
  2. end.

A few things to notice:

  • This is about as simple a job as possible.
  • The job is only using one node, and one CPU on that node. Plenty of room for other users to run their jobs in parallel.
#!/bin/sh
#SBATCH --job-name hello-world
#SBATCH --nodes=1 
#SBATCH -c 1
echo "hello world!"

Example Two

This job will do the following:

  1. Echo some information about the current node (printing it to the output file).
  2. Print the current machine hostname (to the output file, again).
  3. sleep for 10 seconds (wait).
  4. Print the current working directory (to the output file, still).
  5. end.

A few things to notice:

  • The job is only using one node, and one CPU on that node. Plenty of room for other users to run their jobs in parallel.
  • The job will notify the user excellent_email_user@earlham.edu via email when it starts, ends, or fails.
  • The job has a max runtime of 20 seconds, so if something goes wrong and it doesn't end sooner, the scheduler will stop it.
#!/bin/sh
#SBATCH --time=20
#SBATCH --job-name hello-world-two
#SBATCH --nodes=1 
#SBATCH -c 1 # ask for one core
#SBATCH --mail-type=BEGIN,END,FAIL 
#SBATCH --mail-user=excellent_email_user@earlham.edu

echo "queue/partition is `echo $SLURM_JOB_PARTITION`"
echo "running on `echo $SLURM_JOB_NODELIST`"
echo "work directory is `echo $SLURM_SUBMIT_DIR`"

/bin/hostname
sleep 10           # Replace this sleep command with your command line. 
/bin/pwd

Example Three

This job will do the following:

  1. Load the python/3.12 module environment on the machine it's running on.
  2. Run the main.py script with the time command (prints information on runtime after the command finishes).
  3. end.

A few things to notice:

  • The job is only using one node, and 30 CPUs on that node.
  • The job will notify the user excellent_email_user@earlham.edu via email when it starts, ends, or fails.
  • The job has a max runtime of 72 hours, so if something goes wrong and it doesn't end sooner, the scheduler will stop it.
#!/bin/bash
#SBATCH --job-name=MNISTDCGAN # Job name
#SBATCH --output=output_%j.txt # Standard output file with job id
#SBATCH --error=error_%j.txt # Standard error file with job id
#SBATCH --time=72:00:00 # Maximum run time
#SBATCH --nodes=1 # Use one node
#SBATCH --cpus-per-task=30 # Request 30 CPU cores (or more, as available)
#SBATCH --mem=16G # Total memory
#SBATCH --mail-type=BEGIN,END,FAIL # Email notifications
#SBATCH --mail-user=username@earlham.edu # Your email address

module load python/3.12 # replace with any python version you want to use
time python main.py # replace with your python file name


After submitting a job slurm captures anything written to stdout and stderr by the programs and when the job completes puts it in a file called slurm-nnn.out (where nnn is the job number) in the directory where you ran sbatch. Use more to view it when you are looking for error messages, output file locations, etc. If you are used to using qpeek, you can instead just run tail -f jobXYZ.out or tail -f jobXYZ.err.

There's some more CPU management information here.

Useful Slurm Commands

Slurm has some other useful commands that you can use to interact with or view jobs that are running.

Commands
Slurm Description
sbatch run/submit a batch job (.sbatch file, see above)
squeue show jobs currently in the queue
scancel cancel a job by its ID
scontrol show nodes show nodes in the cluster
Environment Variables
Slurm Description
$SLURM_JOB_PARTITION the queue/partition you are in
$SLURM_JOB_NODELIST there's no equivalent of the nodes file but there is an environment variable that stores that information
$SLURM_SUBMIT_DIR working directory from which the command was run

Cluster systems to choose from

The cluster.earlham.edu domain consists of clusters (a collection of physical servers linked through a switch to perform high-performance computing tasks with distributed memory) and jumbo servers (nee "phat nodes"; a system comprising one physical server with a high ratio of disk+RAM to CPU, good for jobs demanding shared memory).

Hamilton

hamilton.cluster.earlham.edu

(Built in 2022) Designed for high CPU/RAM jobs. Used often for photogrammetry work via WebODM and CPU-based AI/ML training.

Services running on this machine

Nodes and Hardware

Machine Name Type CPU RAM GPU
h0 Head Node AMD EPYC 24-core 128GB None
h1 Compute Node AMD EPYC 24-core 256GB None
h2 Compute Node AMD EPYC 24-core 256GB None
h3 Compute Node AMD EPYC 24-core 256GB None
h4 Compute Node AMD EPYC 24-core 256GB None
h5 Compute Node AMD EPYC 24-core 256GB None

Faraday

faraday.cluster.earlham.edu

(Built in 2022) Designed for GPU jobs. Used for research, courses, and projects requiring GPU access, computational biophysics simulations, and notebook services.

Services running on this machine

Nodes and Hardware

Machine Name Type CPU RAM GPU
f0 Head Node AMD EPYC 16-core 192GB Nvidia RTX A5000 24GB
f1 Compute Node AMD EPYC 16-core 192GB Nvidia RTX A5000 24GB
f2 Compute Node AMD EPYC 16-core 192GB Nvidia RTX A5000 24GB
f3 Compute Node AMD EPYC 16-core 192GB Nvidia RTX A5000 24GB
f4 Compute Node AMD EPYC 16-core 192GB Nvidia RTX A5000 24GB
f5 Compute Node AMD EPYC 16-core 192GB Nvidia RTX A5000 24GB

Whedon

whedon.cluster.earlham.edu

(Built in 2015) Designed for high CPU/RAM jobs that require long-term processing. Also used for computational Chemistry simulations with WebMO.

Services running on this machine

Nodes and Hardware

Machine Name Type CPU RAM GPU
w0 Head Node 2x Intel Xeon 8-core 256GB None
w1 Compute Node 2x Intel Xeon 8-core 256GB None
w2 Compute Node 2x Intel Xeon 8-core 256GB None
w3 Compute Node 2x Intel Xeon 8-core 256GB None
w4 Compute Node 2x Intel Xeon 8-core 256GB None
w5 Compute Node 2x Intel Xeon 8-core 256GB None
w6 Compute Node 2x Intel Xeon 8-core 256GB None
w7 Compute Node 2x Intel Xeon 8-core 256GB None

Lovelace

lovelace.cluster.earlham.edu

Designed for particularly high RAM jobs. Used often for large scale biology projects, such as alignment and sequence analysis. This machine also features a particularly large amount of disk space for such projects.

Services running on this machine

Nodes and Hardware

Machine Name Type CPU RAM GPU
lovelace Phat Node 2x Intel Xeon 16-core 1000GB None

Cluster software bundle

The cluster dot earlham dot edu servers all run a supported Debian GNU/Linux 12.

All these servers (unless otherwise noted) also feature the following software:

  • Slurm (scheduler): submit a job with sbatch jobname.sbatch, delete it with scancel jobID. Running a job has its own doc section below.
  • Environment modules: run module avail to see available software modules and module load modulename to load one; you may load modules in bash scripts and sbatch jobs as well.

The default shell on all these servers is bash.

The default Python version on all these servers is Python 2.x, but all have at least one Python 3 module with a collection of widely-used scientific computing libraries.

About qsub

Before Slurm we used Torque and its associated software, including qsub. This is now deprecated and should not be used on the Earlham CS cluster systems.

Tested and working 2022

Retrieved from "https://wiki.cs.earlham.edu/index.php?title=Getting_started_on_clusters&oldid=18326"