RASPBERRY PI HACKS – PART 1: BUILDING MPI FOR PYTHON ON A RASPBERRY PI CLUSTER

This article assumes that a raspberry Pi cluster is running the latest Raspbian OS and the MPICH2 interface is built and is operational. (if you haven’t built a cluster and want to , do comment here with your email id/some contact on social media and I can provide the resource and our procedure sheet) Now the conventional way to install the MPI for python (which is called mpi4py) will not work. That is using the command:

 sudo apt-get install python-mpi4py

will install the mpi4py, but when its run to execute, it fails or crashes. This will be observed only by the developers who have installed MPICH2 interface in their cluster. The reason why it crashes is, unknowingly, the command above will install instances of openMPI. OpenMPI is a different interface that clashes with the one that is already installed, MPICH2. A system is usually designed to run only one interface and when there are multiple instances running, it leads to a system failure.

To avoid this failure and the tedious task to restore the operating system back to its previous state, a work around exists. This work around is to build the mpi4py manually on each of the node in the cluster.

The following are the steps to build it:

1.download the mpi4py package.

curl –k –O https://mpi4py.googlecode.com/files/mpi4py-1.3.1.tar.gz

We can use wget instead of curl but I couldn’t find an option that bypasses the certificate issue that hasn’t been resolved by the website maintenance team.

2.Unpack it. And change to that folder.

tar –zxf mpi4py-1.3.1.tar.gz
cd mpi4py-1.3.1.tar.gz

3.Before the build is started, it is important to make sure that all the python development tools are available. This ensures that many important header files like Python.h is present and can be used by the build function.

(This step can be skipped if the python development tools are already installed)

sudo apt-get update –fix-missing
sudo apt-get install python-dev

4.Now, we can build the package.

cd mpi4py-1.3.1.tar.gz
sudo python setup.py build  –mpicc=/usr/local/mpich2/bin/mpicc

few things that have to be noted here:

The option –mpicc is used to provide the build file the location of the MPI compiler
The option –mpicc has to be used only if the location of that compiler doesn’t already exist in the system path.
The path /usr/local/mpich2/bin/mpicc is the location on my node, where the mpich2 is built. It might not be the same for everyone and so that has to be replaced with the path, where mpicc is located in that system. The only thing now left do is to install the build.to install change working directory to mpi4py:
```
cd mpi4py
```
After shifting to this directory, run the command :
```
sudo python setup.py install
```

Once this is done, repeat the process in every other node in the cluster. Then the demo program helloworld.py can be run to test if mpi4py is installed on all the node successfully and is running correctly.

If the nodes of the cluster aren’t already built, then the easier way to do it would be to perform the above procedure on one node and read the entire image of the OS and write it into the SD cards of each of the other node. This would eliminate building of mpi4py package on each node individually.