Difference between revisions of "Cluster:Modules"

From Earlham CS Department
Jump to navigation Jump to search
(Notes)
(Multiple Repositories)
Line 9: Line 9:
 
* OpenMPI: <code>./configure --prefix=/cluster/software/modules-sw/openmpi/1.3.1/$ARCHPATH && make</code>
 
* OpenMPI: <code>./configure --prefix=/cluster/software/modules-sw/openmpi/1.3.1/$ARCHPATH && make</code>
  
== Multiple Repositories ==
+
== E-mails from Skylar ==
  
 
<pre>
 
<pre>
Line 23: Line 23:
 
the master repository.
 
the master repository.
 
</pre>
 
</pre>
 +
 +
<pre>
 +
I've set up two bobsced meta-modules, one for testing and one for
 +
production, in /usr/share/Modules/modulefiles/modules-bobsced. These
 +
point into corresponding modules and software directories in
 +
/cluster/software/bobsced. The modules are called modules-bobsced/prod
 +
and modules-bobsced/test. The .modulerc file in
 +
/usr/share/Modules/modulefiles/modules-bobsced sets the default to be
 +
modules-bobsced/prod, so just running "module load modules-bobsced" will
 +
load the production module.
 +
 +
When I build software, I usually use the testing module (in this case
 +
modules-bobsced/test), and install into the testing software repository.
 +
Once I make sure the software is working, I'll copy the software
 +
directories and module files into the production directories.</pre>
 +
 +
<pre>
 +
I forgot to mention how to build software for the meta-module. Where the
 +
main CCG module (modules-ccg) would use something like this
 +
 +
./configure --prefix=$MOD_CCGSW/sw-name/sw-version/$ARCHPATH
 +
 +
software in modules-bobsced would use this
 +
 +
./configure --prefix=$MOD_BSSW/sw-name/sw-version/$ARCHPATH
 +
 +
$MOD_BSSW points to /cluster/software/bobsced/modules-sw-test for the
 +
testing module, and /cluster/software/bobsced/modules-sw for production.</pre>
  
 
== Notes ==
 
== Notes ==

Revision as of 11:15, 6 October 2009

Software build options

  ARCHPATH=`uname -s`/`/cluster/software/os_release`/`uname -p`
  • Tcl: ./configure --prefix=/cluster/software/modules-sw/tcl/8.5.7/$ARCHPATH --enable-shared && make
  • Modules: ./configure --prefix=/cluster/software/modules-sw/modules/3.2.7/$ARCHPATH --with-tcl=/cluster/software/modules-sw/tcl/8.5.7/$ARCHPATH/lib --with-static=yes && make
    • Had to remove bash_completion from init/Makefile.
    • After installation, changed the version from 3.2.6->3.2.7 in the init directory, and /usr/share/Modules to /cluster/software/modules-sw/modules/3.2.7/$ARCHPATH/Modules
  • OpenMPI: ./configure --prefix=/cluster/software/modules-sw/openmpi/1.3.1/$ARCHPATH && make

E-mails from Skylar

That's still doable with multiple modules repositories. We can setup
meta-modules that don't actually point you at software but alter how
modules itself behaves. The master meta-module would prepend MODULEPATH
with (say) "/cluster/software/modules" for software that's the same
across all the clusters, and modules-bobsced would prepend MODULEPATH
with "/cluster/bobsced/software/modules". Just like with PATH, the first
hit within a module repository is used. If Perl is only in the master
software repository it'll be grabbed from there, but if OpenMPI is in
both it'll be grabbed from the bobsced repository since it comes before
the master repository.
I've set up two bobsced meta-modules, one for testing and one for
production, in /usr/share/Modules/modulefiles/modules-bobsced. These
point into corresponding modules and software directories in
/cluster/software/bobsced. The modules are called modules-bobsced/prod
and modules-bobsced/test. The .modulerc file in
/usr/share/Modules/modulefiles/modules-bobsced sets the default to be
modules-bobsced/prod, so just running "module load modules-bobsced" will
load the production module.

When I build software, I usually use the testing module (in this case
modules-bobsced/test), and install into the testing software repository.
Once I make sure the software is working, I'll copy the software
directories and module files into the production directories.
I forgot to mention how to build software for the meta-module. Where the
main CCG module (modules-ccg) would use something like this

./configure --prefix=$MOD_CCGSW/sw-name/sw-version/$ARCHPATH

software in modules-bobsced would use this

./configure --prefix=$MOD_BSSW/sw-name/sw-version/$ARCHPATH

$MOD_BSSW points to /cluster/software/bobsced/modules-sw-test for the
testing module, and /cluster/software/bobsced/modules-sw for production.

Notes

  • . /cluster/software/modules.sh gets it working
  • /cluster/software/modules is where the actual configuration files for each module (ie each piece of software) is stored in text files
  • /cluster/software/modules-sw is where the common source code gets installed to (not bobsced-specific)