From Montana Tech Computer Science Department
Jump to: navigation, search

BeoMPI (based on MPICH) is the default Message Passing Interface (MPI) on the system. MPICH2[1], MVAPICH2[2], and OpenMPI[3] are available and easily selected with the env-modules package that allow users to change their environment variables from predefined files (currently only the gnu compiler is loaded). To see what modules are available:

module avail

and to load a module:

module load mpich2/gnu

MPICH2 does not use the InfiniBand (IB) Network, but MVAPICH2 does. Our preliminary benchmarking tests show the best performance with MVAPICH2.

Once the MPI environment is set, the corresponding mpicc and mpif90 are available. For MPICH2 and MVAPICH2, these are wrappers that use the gnu 4.4.7 compilers distributed with the Centos 6.5 are in /usr/bin. (The gcc 4.7 compiler is available in /usr/local/bin.) The OpenMPI compiler wrappers use the gnu 4.7.1 compilers.

Note OpenMPI will use IB with the mca flag:

mpirun --mca btl openib,sm,self -np ....


To compile a C program call mpiprog.c using MPICH2:

module load mpich2/gnu
mpicc mpiprog.c -o mpiprog

To run using torque create a script mpijob with:

#PBS -l nodes=2:ppn=32
#PBS -d /home/mtech/username
#PBS -S /bin/bash
#PBS -m e
#PBS -l walltime=01:00:00
module load mpich2/gnu
mpirun -np 64 --hostfile $PBS_NODEFILE mpiprog

Then submit to moab

msub mpijob