Difference between revisions of "Cluster:Modules"
Jump to navigation
Jump to search
(→Multiple Repositories) |
(→E-mails from Skylar) |
||
Line 51: | Line 51: | ||
$MOD_BSSW points to /cluster/software/bobsced/modules-sw-test for the | $MOD_BSSW points to /cluster/software/bobsced/modules-sw-test for the | ||
testing module, and /cluster/software/bobsced/modules-sw for production.</pre> | testing module, and /cluster/software/bobsced/modules-sw for production.</pre> | ||
+ | |||
+ | <pre> | ||
+ | > So... the modules "software" for a particular modules setup is also | ||
+ | > > called a module? And we have three of these - ccg, bobsced-test, and | ||
+ | > > bobsced-prod? | ||
+ | |||
+ | Right. The module files define what changes to make to the user's | ||
+ | environment. The software installations are stored separately in the -sw | ||
+ | directories. This is necessary because the module command will look for | ||
+ | the modules magic signature (#%Module1.0###) in every file in every | ||
+ | directory listed in MODULEPATH, which can slow things down. | ||
+ | |||
+ | > How do I tell it that I want to have it use production or testing for | ||
+ | > > the variable $MOD_BSSW? | ||
+ | > > | ||
+ | |||
+ | To load the production module, you'd do "module load | ||
+ | modules-bobsced/prod". To see what you actually have loaded do "module | ||
+ | list", and to see what the module actually does do "module disp | ||
+ | modules-bobsced/prod". | ||
+ | </pre> | ||
== Notes == | == Notes == |
Revision as of 13:21, 7 October 2009
Software build options
ARCHPATH=`uname -s`/`/cluster/software/os_release`/`uname -p`
- Tcl:
./configure --prefix=/cluster/software/modules-sw/tcl/8.5.7/$ARCHPATH --enable-shared && make
- Modules:
./configure --prefix=/cluster/software/modules-sw/modules/3.2.7/$ARCHPATH --with-tcl=/cluster/software/modules-sw/tcl/8.5.7/$ARCHPATH/lib --with-static=yes && make
- Had to remove bash_completion from init/Makefile.
- After installation, changed the version from 3.2.6->3.2.7 in the init directory, and /usr/share/Modules to /cluster/software/modules-sw/modules/3.2.7/$ARCHPATH/Modules
- OpenMPI:
./configure --prefix=/cluster/software/modules-sw/openmpi/1.3.1/$ARCHPATH && make
E-mails from Skylar
That's still doable with multiple modules repositories. We can setup meta-modules that don't actually point you at software but alter how modules itself behaves. The master meta-module would prepend MODULEPATH with (say) "/cluster/software/modules" for software that's the same across all the clusters, and modules-bobsced would prepend MODULEPATH with "/cluster/bobsced/software/modules". Just like with PATH, the first hit within a module repository is used. If Perl is only in the master software repository it'll be grabbed from there, but if OpenMPI is in both it'll be grabbed from the bobsced repository since it comes before the master repository.
I've set up two bobsced meta-modules, one for testing and one for production, in /usr/share/Modules/modulefiles/modules-bobsced. These point into corresponding modules and software directories in /cluster/software/bobsced. The modules are called modules-bobsced/prod and modules-bobsced/test. The .modulerc file in /usr/share/Modules/modulefiles/modules-bobsced sets the default to be modules-bobsced/prod, so just running "module load modules-bobsced" will load the production module. When I build software, I usually use the testing module (in this case modules-bobsced/test), and install into the testing software repository. Once I make sure the software is working, I'll copy the software directories and module files into the production directories.
I forgot to mention how to build software for the meta-module. Where the main CCG module (modules-ccg) would use something like this ./configure --prefix=$MOD_CCGSW/sw-name/sw-version/$ARCHPATH software in modules-bobsced would use this ./configure --prefix=$MOD_BSSW/sw-name/sw-version/$ARCHPATH $MOD_BSSW points to /cluster/software/bobsced/modules-sw-test for the testing module, and /cluster/software/bobsced/modules-sw for production.
> So... the modules "software" for a particular modules setup is also > > called a module? And we have three of these - ccg, bobsced-test, and > > bobsced-prod? Right. The module files define what changes to make to the user's environment. The software installations are stored separately in the -sw directories. This is necessary because the module command will look for the modules magic signature (#%Module1.0###) in every file in every directory listed in MODULEPATH, which can slow things down. > How do I tell it that I want to have it use production or testing for > > the variable $MOD_BSSW? > > To load the production module, you'd do "module load modules-bobsced/prod". To see what you actually have loaded do "module list", and to see what the module actually does do "module disp modules-bobsced/prod".
Notes
. /cluster/software/modules.sh
gets it working/cluster/software/modules
is where the actual configuration files for each module (ie each piece of software) is stored in text files/cluster/software/modules-sw
is where the common source code gets installed to (not bobsced-specific)