Modules Environment

What is the main purpose of the Modules environment?

Environment modules provide a great way to easily customize your shell's environment (PATH, MANPATH, INCLUDE, LD_LIBRARY_PATH, etc.) on the fly. The Modules environment allows you to cleanly set and unset all paths and environment variables, which you need in order to use available (and sometimes conflicting) software packages on the HPC system, by simply loading or unloading the corresponding module files. The software packages are thereby differentiated by the following categories:

  1. Application-Software
  2. Compiler
  3. Parallel-Environments
  4. Libraries
  5. Debugging-and-Profiling / Development

Within this structure the majority of installed third party software is contained (e.g. the Intel-compiler) in the form of application/version modulefiles (e.g. intel/11.1). To load or unload a module, respectively modulefile, simply issue the command

module [un]load application/version

OS specific software, such as the currently supported Gnu Compiler Collection (GCC), is naturally not provided by means of the Modules environment, as all binaries, libraries, etc. for such a packet are already contained within the default system paths.

Setting up the Modules environment

The Modules environment will automatically be set during your cluster login. While the general usage is quite self-explanatory, there are the following few usage guidelines to follow:

Setup for a specific compiler

With the exception of the compiler modules, all modulefiles are constructed in such a way, as to set up the environment in accordance with a preferred compiler. Setting the preference to a specific compiler, such as e.g. the Intel compiler is simple: just load the respective compiler module beforehand. The following two command lines

module load intel/11.1
module load netcdf/4.0.1

will assure that everything is set up for the NetCDF 4.0.1 libraries in accordance with the Intel 11.1 compiler. If no special compiler module is loaded, the OS specific GCC is chosen as the preferred compiler.

If for some reason you need to set up a specific module for another compiler than the preferred one, you can enforce this behavior by setting the environment variable

PREFERRED_MC={gnu-$HPC_SYSTEM|intel[-version]|pgi[-version]}

where PREFERRED_MC simply stands for Preferred Module Compiler. So in case you want as preferred compiler the Intel compiler, but for some reasons the netcdf libraries for the system specific Gnu compilers, the following command lines will lead to the desired result:

module load intel/11.1
export PREFERRED_MC=gnu-$HPC_SYSTEM
module load netcdf/4.0.1
unset PREFERRED_MC
module load further_modules

Modules environment and batch jobs

There are (among presumably a myriad of other viable methods) two proposed ways to ensure that the required modules are available for your batch jobs:

  1. You can put your required modules into your .bashrc file within your home directory (module load module_name). This is the preferred method if you use the same modulefiles for all your job runs.
  2. The second and more flexible way is to load a certain module environment as part of your job. This method is recommended if you have to use different environments for different job runs.
    (Please note that this procedure is currently broken for parallel jobs with processes on slave nodes on some of the ZID HPC systems)

    The following script file script.sh serves as an example:

      #!/bin/bash
    module load module_name_1 module load module_name_2 ... program_name <parameters>


    Assure that script.sh has execution rights (chmod +x script.sh).

Note: If you want to submit parallel jobs using the SGE (Son of Grid Engine, formerly Sun Grid Engine) batch system please be aware that by default SGE does not export your actual environment to the remote cluster nodes. More specifically, modules loaded interactively on the master node (e.g. login.leo3, lcc) are not transferred to the compute nodes when a job is submitted with SGE. Method number 1 is the recommended approach to that problem.

Working with the Module Environment

There are a number of sub-commands, which can be supplied to the module command. The most frequently used ones are briefly discussed in the following sections.

avail

With the subcommand avail the list of available modules on the system is printed in the form:

[user@login-node]$ module avail

----------- /path_to_module_categories/{Application-Software|Compiler|...} -----------

application1/version1   application2/version2   application3/version3   ...

display

With the subcommand display you can see what a given modulefile will do to your environment. This way, you can also find out about where application binaries are loaded, respectively, the library paths for linking your own programs. For your convenience and for portability between our HPC systems we provide special environment variables prefixed with UIBK_ for exactly that purpose.

The following example illustrates the display subcommand:

[user@login-node]$ module display intel/11.1

-------------------------------------------------------------------
/usr/site/hpc/modules/leo2/Compiler/intel/11.1:

conflict         intel pgi 
setenv           UIBK_INTEL_BIN /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/bin/intel64 
setenv           UIBK_INTEL_INC /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/include/intel64 
setenv           UIBK_INTEL_LIB /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/lib/intel64 
prepend-path     PATH /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/bin/intel64 
prepend-path     INCLUDE /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/include 
prepend-path     INCLUDE /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/include/intel64 
prepend-path     LD_LIBRARY_PATH /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/lib/intel64 
prepend-path     MANPATH /usr/site/hpc/x86_64/generic/intel/compiler/11.1/current/man/en_US 
prepend-path     INTEL_LICENSE_FILE /usr/site/hpc/x86_64/generic/intel/license/intel-flexlm.lic 
setenv           CC icc 
setenv           CXX icpc 
setenv           FC ifort 
setenv           IDB idb 
-------------------------------------------------------------------

load

With the subcommand load you can load one or more of the available module files:

[user@login-node]$ module load intel/11.1 netcdf/4.0.1

  Loading intel/11.1
  Loading netcdf/4.0.1 for compiler intel-11.1

list

With the subcommand list you get the list of all currently loaded module files:

[user@login-node]$ module list

Currently Loaded Modulefiles:
  1) intel/11.1     2) netcdf/4.0.1

unload/purge

With the subcommand unload you can unload one or more of the loaded module files (see list of loaded modules above):

[user@login-node]$ module unload intel/11.1 netcdf/4.0.1

  Unloading intel/11.1
  Unloading netcdf/4.0.1 for compiler intel-11.1

Similarly, with the command

[user@login-node]$ module purge

all loaded modules are unloaded at once.

help

The subcommand help on its own delivers information about module usage itself. When adding a specific module to the help subcommand, some more information about this module is displayed:

[user@login-node]$ module help netcdf/4.0.1

----------- Module Specific Help for 'netcdf/4.0.1' ---------------

NetCDF (network Common Data Form) is a set of software libraries and machine-independent
data formats that support the creation, access, and sharing of array-oriented scientific
data. Please load this module subsequently to any Matlab module!!!

Additional information about the Modules environment

For further information concerning the Modules environment please have a look at the man pages (man module).
The sources and further documentation can be found at the SourceForge website