Using PBS/Maui

• Paths to commands:
  • /usr/local/{bin,sbin} for PBS.
  • /usr/local/maui/{bin,sbin} for Maui.

• Commands you'll probably use:
  • qsub: Submit a job to a PBS queue.
  • showq: Shows jobs Maui can schedule.
  • qstat: Shows jobs that PBS advertises.
  • canceljob: Cancels jobs.
  • qhold: Holds a job from execution.

• Commands have man pages.

• To submit a job to PBS, you'll need to write a shell script wrapper around it and execute it with qsub on b0. There are commented option to PBS that qsub will interpret. Here's an example script:

#!/bin/sh
#PBS -N primes_block
#PBS -o /cluster/home/skylar/athena/src/primes4/out.txt
#PBS -e /cluster/home/skylar/athena/src/primes4/err.txt
#PBS -q batch
#PBS -m abe

lamboot -v /cluster/home/skylar/athena/lam-bhost.def
mpirun C /cluster/home/skylar/athena/src/primes4/primes_block -n 1000000 -s 500
lamhalt
exit $!

-N gives the job name. -o tells PBS where to copy STDOUT to. -e tells PBS where to copy STDERR to. -q tells PBS which queue to submit the job to. -m abe tells PBS what mail options to use. abe enables all mail information.

• To submit that job, you might run something like this on b0:

qsub -l nodes=8,cput=0:30:0 ./primes_pbs.sh
This will request 8 nodes for a total of 30 minutes CPU time.

• For SMP nodes, use -l ppn=n, where n is the number of processors to request in a machine. For example, use 2 to request dual-processor nodes.

• More information can be found here and here.