MPI for C/C++

MPI stands for Message Passing Interface, it is the standard programming tool for message-passing programs. It replaces the good-old pvm.
We use the LAM/MPI implementation. This implementation seems not to exist for Windows. But since MPI is a standard inferface, any MPI program can be recompiled with a different implementation.
 See also:  MPI Troubleshooting

 Online documentation

  MPI homepage
The mpi reference manual is also available at the lab

LAM/MPI homepage
    User Guide

 Communication primitives for C/C++

MPI Quick Reference page 1
MPI Quick Reference page 2

   Compile your MPI-program

Compile with mpicc for C programs and mpic++ for C++, a wrapper compiler that adds the correct compiler & linker flags.

 Run & debug your parallel program (see LAM/MPI user guide)

On our LINUX system: Computers you can use to run in parallel: parallel1, parallel2 ... parallel7 (Some might not be running)
  1. First start mpi:
  2. To start your parallel program:
  3. To debug a process or redirect the output of the slaves: use a script                 However, this does not work with VNC...

Terminate:

Don't forget to halt mpi with the lamhalt command when you stopped experimenting!!
Otherwise your mpi-processes keep on running! Also when you log out!
 

Cluster   ONLY ACCESSIBLE FROM OUR LINUX SYSTEM 

Do your final experiments (performance measurements) on the cluster (ask first for permission to your assistent):

   Program Example

Download the code

Download the code all at once with this compressed archive file: mpiExample.tgz, extract the files in a new directory with command: tar -zxf mpiExample.tgz
The files:

Run the program

    1. start MPI with lamboot hostfile.txt
    2. create the makefile starting from the project file problem.pro with qmake and compile it
    3. run your parallel program
    4. try to understand the program (documentation)
    5. Study it:
Check our Quicklist and  page!!

Exercises

    1. Do a small performance analysis:
    2. Optimize the communication by choosing a more appropriate MPI communication operation, check whether it improves performance.
    3. Change the task of the parallel program to Is8398551DivisibleByValue. How is the performance now? why?
    4. Measure the communication time and the ratio communication versus computation
    5. Solve the load imbalance problem!
    6. eg: divide the given range in a great number (> p) of subranges. Send the first subranges to the slaves, send the next subranges whenever a slave has finished with its subrange.

 Program Example with automatic EPPA instrumentation

Download the code

Download the code all at once with this compressed archive file: mpiEPPAExample.tgz, extract the files in a new directory with command: tar -zxf mpiEPPAExample.tgz
The program is similar to the previous one, only:
  1. standardlib is ommited, since it is integrated in the par library.
  2. We check the status of the message to retrieve the array size (functions send_array & receive_array).
  3. Error handling is changed with MPI_Comm_set_errhandler (after the MPI_Init), now the MPI will simply return when an error occurs and the return code should be checked for errors. We do thsi with CheckForMpiError to print more detailed information about the error.
  4. Additional MPI_Pcontrol statements to provide extra information to EPPA (but this is not really necessary).
  5. The program is linked with the par library (this automatically instruments your MPI program) & mysql library (to write your results to our MYSQL database)
  6. The number of basic operations (variable gTaskNbrCalculations) is counted in the functions of the tasks.cc file, this is passed to EPPA.

On our LINUX system
EPPA is installed under the $PARDIR directory, which is in your path, so just type EPPA to run it. It shows all of your experiments.
With ViewExp <experiment ID> you can visualize one experiment. With ViewExp your latest experiment is shown.

On another system (with QT, mpi and mysql), EPPA can be installed
  See EPPA User Documentation

 More examples