For detailed information about Slurm, please take a look on the :ref:`Quick Introduction <quickintro>` and :ref:`Batch system <batchsystem>` page.
With Slurm 17.11 support for Heterogeneous Jobs was introduced. This allows to spawn a job across multiple partitions of a cluster, and across different Modules of our Supercomputers. See the official Slurm documentation (SlurmHetJob_) for additional informations on this feature.
Executing applications in a modular environment, especially when different Modules have different architectures or the dependencies of programs are not uniform, can be a challenging tasks.
**Uniform Architecture and Dependencies**
As long as the Architecture of the given modules are uniform and there are not mutually exclusive dependencies for the binaries that are going to be executed, one can rely on the ``module`` command. Take a look on the :ref:`Quick Introduction <quickintro>` if ``module`` is new for you.
.. code-block:: none
#!/bin/bash -x
#SBATCH ...
module load [...]
srun ./prog1 : ./prog2
**Non Uniform Architectures and Mutual Exclusive Dependencies**
A tool called ``xenv`` was implement to ease the task of loading modules for heterogeneous jobs. For details on supported command line arguments, execute ``xenv -h`` on the given system.
When the nodes of a job belong to different interconnects and MPI communication is used, bridging has to take place. To support this workflow, e.g. run a job on a Cluster with Infiniband and a Booster with OmniPath, a Gateway Daemon (psgwd, ParaStation Gateway Daemon) was implemented that takes care of moving packages across fabrics.
To request gateway nodes for a job the mandatory option --gw_num has to be specified. In addition, communication with the psgwd has to be ensured via loading the software module **pscom-gateway** either via ``xenv`` or the ``module`` command.
To start an interactive pack job using two gateway nodes the following command can be used: