Cluster: New BobSCEd Install Log - Revision history

Kay: /* Cluster Image Review */

2010-04-22T00:16:38Z

Cluster Image Review

← Older revision		Revision as of 00:16, 22 April 2010
Line 176:		Line 176:
	\| align="center" style="background:#f0f0f0;"\|'''Notes'''		\| align="center" style="background:#f0f0f0;"\|'''Notes'''
	\|-		\|-
	\| mpich1\|\|OK, except allocates 1 process to bs0\|\|OK, except allocates 1 process to bs0\|\|OK\|\|OK\|\|/mounts/bobsced/usr/local/modules-sw/mpich1/1.2.7p1/share/machines.LINUX\|\|Creates temporary file PIxxxxx while running under qsub; MPI does not respect qsub ~~# of~~ nodes given		\| mpich1\|\|OK, except allocates 1 process to bs0\|\|OK, except allocates 1 process to bs0\|\|OK\|\|OK\|\|/mounts/bobsced/usr/local/modules-sw/mpich1/1.2.7p1/share/machines.LINUX\|\|Creates temporary file PIxxxxx while running under qsub; MPI does not respect qsub nodes given, could get around this by creating machinefile at runtime
	\|-		\|-
	\| mpich2-osc\|\|N/A\|\|N/A\|\|OK, MUST specify # nodes, ppn\|\|OK, MUST specify # nodes, ppn\|\|Generated by PBS\|\|Currently uses OSC's pbs-specific mpiexec (cannot be run outside of qsub)		\| mpich2-osc\|\|N/A\|\|N/A\|\|OK, MUST specify # nodes, ppn\|\|OK, MUST specify # nodes, ppn\|\|Generated by PBS\|\|Currently uses OSC's pbs-specific mpiexec (cannot be run outside of qsub)

Kay: /* Modules Software */

2010-04-22T00:15:35Z

Modules Software

← Older revision		Revision as of 00:15, 22 April 2010
Line 17:		Line 17:
	* MPICH1 installed with <code>./configure --prefix=/mounts/bobsced/usr/local/modules-sw/mpich1/1.2.7p1</code>		* MPICH1 installed with <code>./configure --prefix=/mounts/bobsced/usr/local/modules-sw/mpich1/1.2.7p1</code>
	** Make note: to uninstall - /mounts/bobsced/usr/local/modules-sw/mpich1/1.2.7p1/sbin/mpiuninstall		** Make note: to uninstall - /mounts/bobsced/usr/local/modules-sw/mpich1/1.2.7p1/sbin/mpiuninstall
	** It looks to /mounts/bobsced/usr/local/modules-sw/mpich1/1.2.7p1/share/machines.ARCH instead of just running on the host machine if run without a machine file. I removed this file to force people to use a machinesfile... that's going to have to be generated from PBS as part of the qsub script.		** It looks to /mounts/bobsced/usr/local/modules-sw/mpich1/1.2.7p1/share/machines.ARCH instead of just running on the host machine if run without a machine file. I removed this file to force people to use a machinesfile... that's going to have to be generated from PBS as part of the qsub script. <font color="green">Gave up and added it back in until I finish writing a qsub script that incorporates it</font>
	* MPICH2 installed with		* MPICH2 installed with
	:<code>./configure --prefix=/mounts/bobsced/usr/local/modules-sw/mpich2/2.1.2 --enable-cxx --enable-f90 --enable-f77 --enable-threads=multiple --with-thread-package=posix</code>		:<code>./configure --prefix=/mounts/bobsced/usr/local/modules-sw/mpich2/2.1.2 --enable-cxx --enable-f90 --enable-f77 --enable-threads=multiple --with-thread-package=posix</code>

Kay: /* Cluster Image Review */

2010-04-21T23:49:18Z

Cluster Image Review

← Older revision		Revision as of 23:49, 21 April 2010
Line 182:		Line 182:
	\| mpich2 \|\|OK, must setup mpd ring\|\|OK, must set up mdp ring\|\|OK, must set up mdp ring\|\|OK, must set up mdp ring\|\|N/A (mpd ring)\|\|Requires chmod 600 .mpd.conf in home directory with MPD_SECRETWORD=somevalue; setup ring with <code><nowiki>sort $PBS_NODEFILE \| uniq -c \| awk '{print $2":"$1}' > /cluster/home/kwanous/tmp/mpd.nodes; mpdboot -f /cluster/home/kwanous/tmp/mpd.nodes -n 2</nowiki></code>		\| mpich2 \|\|OK, must setup mpd ring\|\|OK, must set up mdp ring\|\|OK, must set up mdp ring\|\|OK, must set up mdp ring\|\|N/A (mpd ring)\|\|Requires chmod 600 .mpd.conf in home directory with MPD_SECRETWORD=somevalue; setup ring with <code><nowiki>sort $PBS_NODEFILE \| uniq -c \| awk '{print $2":"$1}' > /cluster/home/kwanous/tmp/mpd.nodes; mpdboot -f /cluster/home/kwanous/tmp/mpd.nodes -n 2</nowiki></code>
	\|-		\|-
	\| openmpi 1.*\|\|OK for running on one node\|\|OK for running on one node\|\|Uses ~~PBS_NODE~~ automatically\|\|Uses ~~PBS_NODE~~ automatically\|\|		\| openmpi 1.*\|\|OK for running on one node\|\|OK for running on one node\|\|Uses PBS_NODES automatically\|\|Uses PBS_NODES automatically\|\|
	\|}		\|}

Kay: /* Cluster Image Review */

2010-04-21T23:48:49Z

Cluster Image Review

← Older revision		Revision as of 23:48, 21 April 2010
Line 182:		Line 182:
	\| mpich2 \|\|OK, must setup mpd ring\|\|OK, must set up mdp ring\|\|OK, must set up mdp ring\|\|OK, must set up mdp ring\|\|N/A (mpd ring)\|\|Requires chmod 600 .mpd.conf in home directory with MPD_SECRETWORD=somevalue; setup ring with <code><nowiki>sort $PBS_NODEFILE \| uniq -c \| awk '{print $2":"$1}' > /cluster/home/kwanous/tmp/mpd.nodes; mpdboot -f /cluster/home/kwanous/tmp/mpd.nodes -n 2</nowiki></code>		\| mpich2 \|\|OK, must setup mpd ring\|\|OK, must set up mdp ring\|\|OK, must set up mdp ring\|\|OK, must set up mdp ring\|\|N/A (mpd ring)\|\|Requires chmod 600 .mpd.conf in home directory with MPD_SECRETWORD=somevalue; setup ring with <code><nowiki>sort $PBS_NODEFILE \| uniq -c \| awk '{print $2":"$1}' > /cluster/home/kwanous/tmp/mpd.nodes; mpdboot -f /cluster/home/kwanous/tmp/mpd.nodes -n 2</nowiki></code>
	\|-		\|-
	\| openmpi 1.*\|\|OK ~~(can send machinefile)~~\|\|OK ~~(can send machinefile)\|\|All runs~~ on one node ~~(can send machinefile)~~\|\|\|\|		\| openmpi 1.*\|\|OK for running on one node\|\|OK for running on one node\|\|Uses PBS_NODE automatically\|\|Uses PBS_NODE automatically\|\|
	\|}		\|}

Line 204:		Line 204:
	mpiexec -np 9 /cluster/home/kwanous/a.out		mpiexec -np 9 /cluster/home/kwanous/a.out
	mpdallexit		mpdallexit

			rm -rf /cluster/home/kwanous/tmp/mpd.nodes
	</pre>		</pre>

Line 220:		Line 222:

	'''Openmpi'''		'''Openmpi'''
			<pre>#!/bin/bash
			#PBS -N testopenmpi
			#PBS -l cput=00:60:00
			#PBS -l nodes=2:ppn=4

			hostname
			. /etc/profile.d/modules.sh
			module load modules modules-init modules-bobsced
			module load openmpi/1.3.1
			mpirun -np 9 /cluster/home/kwanous/a.out</pre>

Kay: /* Cluster Image Review */

2010-04-21T23:34:39Z

Cluster Image Review

@@ Line 182: / Line 182: @@
 | mpich2 ||OK, must setup mpd ring||OK, must set up mdp ring||OK, must set up mdp ring||OK, must set up mdp ring||N/A (mpd ring)||Requires chmod 600 .mpd.conf in home directory with MPD_SECRETWORD=somevalue; setup ring with <code><nowiki>sort $PBS_NODEFILE | uniq -c | awk '{print $2":"$1}' > /cluster/home/kwanous/tmp/mpd.nodes; mpdboot -f /cluster/home/kwanous/tmp/mpd.nodes -n 2</nowiki></code>
 |-
-| openmpi 1.3.1||||||||||
+| openmpi 1.*||OK (can send machinefile)||OK (can send machinefile)||All runs on one node (can send machinefile)||||
 |}

Kay at 23:25, 21 April 2010

2010-04-21T23:25:41Z

← Older revision		Revision as of 23:25, 21 April 2010
Line 1:		Line 1:

	The source code for anything installed locally is in <code>/usr/local/src</code>. The source for anything installed on NFS is in <code>/mounts/bobsced/usr/local/src</code>.		The source code for anything installed locally is in <code>/usr/local/src</code>. The source for anything installed on NFS is in <code>/mounts/bobsced/usr/local/src</code>.

Kay: /* Modules Software */

2010-04-13T18:10:45Z

Modules Software

← Older revision		Revision as of 18:10, 13 April 2010
Line 11:		Line 11:
	* 1.3.1 - configured with <code>./configure --prefix=/mounts/bobsced/usr/local/modules-sw/openmpi/1.3.1/ --enable-mpi-threads --with-openib</code>		* 1.3.1 - configured with <code>./configure --prefix=/mounts/bobsced/usr/local/modules-sw/openmpi/1.3.1/ --enable-mpi-threads --with-openib</code>
	* 1.3.3 same		* 1.3.3 same
			* 1.4.1 same

	'''MPICH'''		'''MPICH'''

Kay: /* Cluster Image Review */

2010-04-13T18:09:19Z

Cluster Image Review

← Older revision		Revision as of 18:09, 13 April 2010
Line 178:		Line 178:
	\| mpich2-osc\|\|N/A\|\|N/A\|\|OK, MUST specify # nodes, ppn\|\|OK, MUST specify # nodes, ppn\|\|Generated by PBS\|\|Currently uses OSC's pbs-specific mpiexec (cannot be run outside of qsub)		\| mpich2-osc\|\|N/A\|\|N/A\|\|OK, MUST specify # nodes, ppn\|\|OK, MUST specify # nodes, ppn\|\|Generated by PBS\|\|Currently uses OSC's pbs-specific mpiexec (cannot be run outside of qsub)
	\|-		\|-
	\| mpich2 \|\|\|\|\|\|\|\|\|\|\|\|Requires chmod 600 .mpd.conf in home directory with MPD_SECRETWORD=somevalue; setup ring with <code><nowiki>sort $PBS_NODEFILE \| uniq -c \| awk '{print $2":"$1}' > /cluster/home/kwanous/tmp/mpd.nodes; mpdboot -f /cluster/home/kwanous/tmp/mpd.nodes -n 2</nowiki></code>		\| mpich2 \|\|OK, must setup mpd ring\|\|OK, must set up mdp ring\|\|OK, must set up mdp ring\|\|OK, must set up mdp ring\|\|N/A (mpd ring)\|\|Requires chmod 600 .mpd.conf in home directory with MPD_SECRETWORD=somevalue; setup ring with <code><nowiki>sort $PBS_NODEFILE \| uniq -c \| awk '{print $2":"$1}' > /cluster/home/kwanous/tmp/mpd.nodes; mpdboot -f /cluster/home/kwanous/tmp/mpd.nodes -n 2</nowiki></code>
	\|-		\|-
	\| openmpi 1.3.1\|\|\|\|\|\|\|\|\|\|		\| openmpi 1.3.1\|\|\|\|\|\|\|\|\|\|

Kay: /* Cluster Image Review */

2010-04-13T17:43:32Z

Cluster Image Review

← Older revision		Revision as of 17:43, 13 April 2010
Line 178:		Line 178:
	\| mpich2-osc\|\|N/A\|\|N/A\|\|OK, MUST specify # nodes, ppn\|\|OK, MUST specify # nodes, ppn\|\|Generated by PBS\|\|Currently uses OSC's pbs-specific mpiexec (cannot be run outside of qsub)		\| mpich2-osc\|\|N/A\|\|N/A\|\|OK, MUST specify # nodes, ppn\|\|OK, MUST specify # nodes, ppn\|\|Generated by PBS\|\|Currently uses OSC's pbs-specific mpiexec (cannot be run outside of qsub)
	\|-		\|-
	\| mpich2 \|\|\|\|\|\|\|\|\|\|\|\|Requires chmod 600 .mpd.conf in home directory with MPD_SECRETWORD=somevalue; setup ring with sort $PBS_NODEFILE \| uniq -c \| awk '{print $2":"$1}' > /cluster/home/kwanous/tmp/mpd.nodes; mpdboot -f /cluster/home/kwanous/tmp/mpd.nodes -n 2		\| mpich2 \|\|\|\|\|\|\|\|\|\|\|\|Requires chmod 600 .mpd.conf in home directory with MPD_SECRETWORD=somevalue; setup ring with <code><nowiki>sort $PBS_NODEFILE \| uniq -c \| awk '{print $2":"$1}' > /cluster/home/kwanous/tmp/mpd.nodes; mpdboot -f /cluster/home/kwanous/tmp/mpd.nodes -n 2</nowiki></code>
	\|-		\|-
	\| openmpi 1.3.1\|\|\|\|\|\|\|\|\|\|		\| openmpi 1.3.1\|\|\|\|\|\|\|\|\|\|

Kay: /* Cluster Image Review */

2010-04-13T17:42:31Z

Cluster Image Review

← Older revision		Revision as of 17:42, 13 April 2010
Line 178:		Line 178:
	\| mpich2-osc\|\|N/A\|\|N/A\|\|OK, MUST specify # nodes, ppn\|\|OK, MUST specify # nodes, ppn\|\|Generated by PBS\|\|Currently uses OSC's pbs-specific mpiexec (cannot be run outside of qsub)		\| mpich2-osc\|\|N/A\|\|N/A\|\|OK, MUST specify # nodes, ppn\|\|OK, MUST specify # nodes, ppn\|\|Generated by PBS\|\|Currently uses OSC's pbs-specific mpiexec (cannot be run outside of qsub)
	\|-		\|-
	\| mpich2 \|\|\|\|\|\|\|\|\|\|\|\|Requires chmod 600 .mpd.conf in home directory with MPD_SECRETWORD=somevalue		\| mpich2 \|\|\|\|\|\|\|\|\|\|\|\|Requires chmod 600 .mpd.conf in home directory with MPD_SECRETWORD=somevalue; setup ring with sort $PBS_NODEFILE \| uniq -c \| awk '{print $2":"$1}' > /cluster/home/kwanous/tmp/mpd.nodes; mpdboot -f /cluster/home/kwanous/tmp/mpd.nodes -n 2
	\|-		\|-
	\| openmpi 1.3.1\|\|\|\|\|\|\|\|\|\|		\| openmpi 1.3.1\|\|\|\|\|\|\|\|\|\|