ABINIT Discussion Forums

Dear Developper,

I recently realized that paral_kgb was automatically set to 1 if the number of cpu was not a multiple of the total number of k-points.

Altought this is in principle a good idea, for the moment the paral_kgb default value is well 0 (deactivated).

The problem is that the algorithm is not stable.

For example a simple ground-state calculation of diamond fail to converged after 200 steps (scf cycle) with the automatic parallelization on 8 cpu (2 cpu at the fft level and 4 at the kpoint level). With the input variable paral_kgb 0 explicitely set in the abinit input file the calculation converges in 13 steps. The calculation was still far faster altough some cpu were idle (due to a wrong choosing of cpu with number of kpoints).

I therefore think that for the moment paral_kgb should always be 0 by default. At some point, when the implementation will be well tested we should indeed change this to a default value of paral_kgb 1.

Do you agree?

Best Regards,

Samuel

Hi Samuel,

I completely agree with you!
Ok we know that paral_kgb 0 presents significant limitations in terms of parallelism but,
on the other hand, this method is compatible with the most important features of abinit,
in particular the post-processing routines that are called at the end of the GS run.

Changing the default value when the number of MPI nodes is not efficient makes sense
all if all the features of the GS part are compatible with the band-G distribution and I don't
think so especially if we consider that the majority of the GS tests validate the band-by-band CG method.

Moreover, as pointed out by Samuel, paral_kgb = 1 is not necessarily faster than the old method when
both eigensolvers can be applied to the same system.
The autoparal feature introduced in Abinit is very useful but, unfortunately, it's hard to predict if
the eigenvalue solver will have troubles to converge or if the hardware/network is suited for that
particular algorithm.
We can try to optimize the parameters of the MPI run once the eigensolver has been select
but we should let the user select the GS algorithm to be used

A possible solution would be to force the specification of paral_kgb in the input file so that no ambiguity occurs
This is the same approach used for other GS variables such as toldfe/tolvrs... so I don't see why we should introduce exceptions.

Cheers,
Matteo

Hey,

This is not so simple as it seems to be.

For years, we have been asked to have an "easy-to-use" abinit on paralell systems.
In the old days (not so old!), we had a lot of keywords to put in the input file to make abinit run; moreover, it often failed because these keywords where incorrectly set.

Now that everything is automatic, a different view appears (more from developpers than from regular users).

So, the question is the following:
How do we change this situation (ABINITT seems having an undesirable behavior) without falling back into the previous (also undesirable) status ?

My proposal:
* We introduce a new value of autoparal that completely deactivates the automatic parallelization (should be zero; but it is already assigned); this new option simply will be equivalent to a return in the finddistrproc routine.
* We set the default value of autoparal to this new option, in such a way that the automatic parallelization will be deactivated.
* If ABINIT is launched on more processors thant nkpt*nsppol, then a WARNING is printed, advicing (strongly) the user to put the autoparal variable.

My opinion is that putting "autoparal 1" is a sufficiently simple action that should not make the use of ABINIT too complicated on parallel systems.

I can do that in the forthcoming days...
What do you think ?

Marc



BTW: I can show severallexamples where lobpcg converges faster than cgwf; it's system dependent...

Dear Marc,

Thank you for your reply.

If I understand correctly you proposal it seems great! I would made a slight modification to it:

Marc proposal:
* We introduce a new value of autoparal that completely deactivates the automatic parallelization (should be zero; but it is already assigned); this new option simply will be equivalent to a return in the finddistrproc routine.
* We set the default value of autoparal to this new option, in such a way that the automatic parallelization will be deactivated.
* If ABINIT is launched on more processors thant nkpt*nsppol, then a WARNING is printed, advicing (strongly) the user to put the autoparal variable.

I think that one can introduce a new value of autoparal that completely deactivates the automatic parallelization and it should be zero. The behavior associated to the already assigned 0 value should be changed to the value 1 and the other pushed by one number.
I totally agree with the two other propositions !

Samuel.

We introduce a new value of autoparal that completely deactivates the automatic parallelization

Fine with this. On the other hand, I think that it's much more intuitive if autoparal=0 means **no automatic parallelization**.
We can change the meaning of the variable in the next major version and we document this change

Note however that we haven't solved yet the issue with the definition of paral_kgb.
Is autoparal allowed to change the value of paral_kgb at run-time?
Is the user expected to specify explicitly the eigensolver he/she wants to use or not?

BTW:
I would really appreciate if we start to rewrite some portions of the GS/DFPT code using a more flexible API.
Passing paral_kgb to a routine just to signal that we may be using FFT-MPI is clearly a design flaw (see fourdp and fourwf)
paral_kgb should drive the computation in the high-level routines. *Simple* algorithms that, for some reason,
have to deal with the FFT parallelism can easily implement it by using just me_g0, comm_fft and some additional table (distribfft_type).
I find very amusing the fact that one finds references to dtset%paral_kgb both in the DFPT and in the GW code albeit
paral_kgb has not meaning in these two contexts!
This won't certainly facilitate the implementation of new algorithms and code reusability

Cheers,
Matteo

+1

This is indeed a great idea to break the old habits and design before coding, for once.