Distributed memory parallelisation

[dbeurle]

This issue tracks the progress towards inclusion of MPI based parallelisation. Before this is included there should be a set of test codes to ensure that the currently used libraries can be enabled with MPI. Once this is completed, the linear solver backends need to be changed and the io implementation will be swapped out to include parallel file io.

For a worthwhile implementation it makes sense to restrict the choice of solvers to a 'fully' distributed such that no particular process has a complete view of the stiffness matrix. This method scales better because no one process has a world view on the problem.

1. Include external c++ wrappers for MPI (mpi-bind)
2. Distributed PaStiX
3. Distributed MUMPS (slight issues with this)
4. CPU Iterative solver suite
5. GPU Iterative solver suite
6. Distributed VTK IO

[dbeurle]

Update:

• External wrapper seems to be working. Some more work and documentation to follow on this when time permits.
• PaStiX 5 is not fully distributed. Each column cannot appear more on more than one processor, so some communication is required for each solution. Either define master-slave relationships for each DoF (globally) before each solve or wait for response from the PaStiX devs on potentially making this more user friendly update mixed distributed matrices will not be supported.
• MUMPS is fully distributed but requires the right hand side to live entirely on the host (not too bad). This will need to be tested and merge this into Eigen if the developers allow it. It would mean very little work in the code and let the heavy lifting be done by the library.
• The distributed CPU solver is working as proof of concept. @annierhea is handling the iterative workload.
• Parallel VTK is working with very minimal changes.
• amgcl is also an option but does not support multiple entries across processors, leaving an adapter class to compute the correct matrices.

To handle this we need to know if the following is possible:

• Assemble sparse_matrix with global indices (gaps in numbering possible)
• Empty rows / cols possible?
• Deletion of rows / cols when shuffling data and accumulating partial matrix coefficients