Pvm Troubleshooting

First identify the type of your problem:

1. I cannot start pvm
2. I cannot add machines in pvm
3. I cannot compile my parallel program
   
4. My master program crashes (runtime)
   
5. My slave program crashes (runtime)
6. while configuring a PVM (a parallel virtual machine)

Troubles starting pvm.

• remove your pvmd.xxxxx file in the local /tmp directory, as explained below
  

Problems with adding machines in pvm.

Solution:

1. It could be that someone restarted that machine in order to use another operating system. Since we do not install the software for operating systems others than Linux, you will not be able to add a machine to a PVM if it is not running under Linux. Or the machine is simply down. Check this by trying to ssh to that machine: ssh machine. The error message No Route To Host indicates that the machine is down.
   
2. Is the name of the machine you are using present in your .rhosts file (see pvm configuration)? If not, then the local machine will not be allowed to start pvmd for you.
3. Always halt your PVM when you finished your tests!
4. If you get very quickly the 'Can't Start pvmd'  response:
   

Problem: I can not add elements of the Crunch cluster to my PVM. It is very strange because it works prefectly for the other machines of the lab.

Solution:

1. Check your .rhosts file please. It should contain crunch. This is the name of the machine that severs as gateway to the cluster or to be more precise that is the name the nodes of Crunch use. You read it correctly. The gateway is called crunch when looking at it from within the cluster and info9 when looking from say... the internet.
2. Another possiblity is that I forgot the add your profile to the crunch-users on the cluster in which case you should e-mail me.

Problem: I have some problems adding node of the cluster this time, it used to work before.

Solution:

1. This problem is again related to the lock files that are left behind on the machines. The situation on the Crunch cluster is somewhat different in that you can not start telnet sessions to the different nodes of the cluster. You can use remote shell however so you do not need to login on the machines to solve you problem! Follow the different steps described underneath. Of course you need to replace the x with the number of the node that is bothering you.
2. Type 'rsh crunchx ls /tmp' in a shell. This should display a list of file in the /tmp directory of that machine. You should find a pvmd.xxxx file where xxxx is your user id. See here if you don not know you user id on our machines.
3. You will now delete this lock file of yours by typing this: 'rsh crunchx rm /tmp/pmvd.xxxx' That should do it.

Compile time problems

• Several include files are reported as missing during compilation.
• All the pvm functions in my program given a error when I compile.
• My program compiles without any problem and yet it does not work correctly.

Solution:

1. You need to use an include statementlike #include <pvm.h>. In that case you need to specify to gcc were it can find the system wide include files. Why? Because this include syntax assumes that the include file is a standard include file. You can specify several path for the include files using a capital i as a command line parameter for gcc. Like this: gcc a_program.c -I/path_to_include_files.

Problem: When linking I get a lot of errors for each of the pvm functions I use.

Solution:

1. These errors are probably called undefined reference. This is a very simple problem, you need to link your program with the pvm library. You can either use the -L command line option or a combination of -L and -l. In the first case you specify the complete path to the library file with -L. Like this: gcc program.c -L/home/pvm3/lib/pvm3. Or you can combine this with -l this is more convenient if you have several library files in one directory. The syntax is then the following one: gcc program.c -L/home/pvm3/lib -lpvm3.

Run time problems with master

• When I start my program I get some strange warnings about the libpvm.

1. You forgot to start a PVM. As a result you program is not able to request services from any PVM and it can not run.
2. You did indeed start a PVM but you start your program on a machine that is not part of the PVM. Use the command hostname in your console and conf in pvm to check this.
   

Runtime problems with slave

• the executable should be in the ~/pvm3/bin/LINUX directory

PVM configuration problems

• PVM does not start anymore.
• An attempt to add a host to a PVM is very slow and fails in the end.
• It is not possible to add some hosts although it was possible before.
• I can not add one or more hosts of the Crunch cluster.
• It seems like I can not add nodes on Crunch anymore.
• I do not know my user identification on the systems of the lab.

  Solution:

1. This topic is related to the problem described in the next section, so please read it the find the real solution to your problem. But keep reading the next point to understand what happened.
2. The machine on which you issued the pvm-command seems to have suffered an abnormal stop in its operations (in other words it crashed or someone stopped it in an improper manner). This inadvertant stop makes it of course impossible for the different programs to remove their lock-files. The symptoms differ from the one of the next section because the lock file is present on the local machine that all.
3. Always stop your PVM when you finished your tests to avoid this kind of problems.