Tips

This page is an attempt to give some very general guidelines on how to use the parallel kernel. These are mostly details and you should not expect anything magic here.

Latency hiding

The idea here is to overlap computation and communication on 1 node as much as possible. The underlying idea is that the communication is happening in the background and can not directly be controlled by the user. The user can do nothing but wait in the case of a blocking receive for instance. So while a program is waiting for a message to arrive this program is doing nothing!

Why not make use of this idle time to run another program. This of course the idea that is exploited in most OSes today. But we can also use it for our parallel programs by putting 2 or more instances of a parallel program on a computing node of a cluster. The OS will schedule the different tasks in such a way that the program waiting for IO will be put to sleep and other programs can do some more useful work. As a result we don't no waste time waiting since we run the other task instead thus hiding the latency of the communication.

In lil-gp you can control the communication between the computing nodes through the exchanges in the input file your provide to the kernel. So maybe, very maybe, you can reorder the exchanges in such a way the maximize the latency hiding. Unfortunately, I can not be more precise on this matter since your exchange topology and the interconnect between the computing nodes in very important here.

Multiple experiments

Evolutionary algorithms have a stochatic component which usually require several runs of the algorithm in orde the acquire representative data. You can abuse the parallel kernel so that you can run several experiments simultaneously. In this case you without using the island model at all. This can be done with the configuration file. In this case it suffices to avoid defining any exchanges at all! What will happen at that moment is that in synchronous mode the kernel will use barrier synchronisation to make sure that all the nodes are busy with the same generation. If you do not like this just, invoke the kernel in asynchronous mode, this shouldn't hurt.

The information provided on this page is copyrighted © by the Free University of Brussels and provided as is.
From more information contact Johan Parent