Page 1 of 1
processor grid layout
Posted: Sat Apr 28, 2012 1:27 am
by jtv
In the routine 51_manage_mpi/initmpi_grid.F90 the array telling mpi how to arrange the processes (mpi_energ%sizecart) is written in Fortran ordering, but I believe the mpi routine (mpi_cart_create) is assuming C ordering. The effect of this is that when using KGB parallelization the FFT processors are not adjacent but rather maximally dispersed. Inverting the arrays in this subroutine (such that spin if the first index and fft is the fourth) seems to show improved speed/scaling.
Re: processor grid layout
Posted: Sun Jun 10, 2012 11:46 pm
by torrent
Dear "jtv",
Thanks for the performance report.
But, before modifying the source code, we have to check that your tests are sufficiently general.
I mean, changing the FFT distribution could have laso an impact on the band distribution.
Could you exactly give us an example of input file demonstrating the performance improvment if with change sizecart in initmpi_grid ?
Could you also give us more details about the architecture you use (computer type, size of nodes, MPI version, ...) ?
Thanks
Marc