Memory usage openmpi configuration

option, parallelism,...

Moderators: fgoudreault, mcote

Forum rules
Please have a look at ~abinit/doc/config/build-config.ac in the source package for detailed and up-to-date information about the configuration of Abinit 8 builds.
For a video explanation on how to build Abinit 7.x for Linux, please go to: http://www.youtube.com/watch?v=DppLQ-KQA68.
IMPORTANT: when an answer solves your problem, please check the little green V-like button on its upper-right corner to accept it.
Locked
JEJohns
Posts: 55
Joined: Sun May 02, 2010 5:30 pm

Memory usage openmpi configuration

Post by JEJohns » Sat Jul 17, 2010 9:11 pm

I'm not sure if this is the right forum to put this in, but it seems like a good first shot. I've been running abinit 6.0.3 on an x86_64_gnu_4.4 quad-core 8gb machine, compiled with mpirun, and everything works well for small systems. For systems that require larger than 2gb of memory with 4 processors, my jobs get killed due to an out of memory issue. When I top on a job that should take 2.5GB of memory, I see that there are 4 abinit processes, each of which has allocated 2.5 GB of virtual memory, and my job crashes during the first iteration. Do I need to have enough memory so that each processor has access to 2.5gb of memory, or is something misconfigured?

mverstra
Posts: 655
Joined: Wed Aug 19, 2009 12:01 pm

Re: Memory usage openmpi configuration

Post by mverstra » Sun Jul 18, 2010 12:17 pm

Hello,

in practice, yes:

1) abinit uses a bit more memory than announced (only the biggest arrays are counted) - in some exotic cases the estimation is completely off (not for normal ground state or DFPT though)

2) almost always, the memory must be _real_ - no counting on linux swap spaces etc... which are usually for programs which are not actively running. If you want to reduce your footprint, check mkmem, or distribute over more processors (not on your machine, though).

matthieu
Matthieu Verstraete
University of Liege, Belgium

JEJohns
Posts: 55
Joined: Sun May 02, 2010 5:30 pm

Re: Memory usage openmpi configuration

Post by JEJohns » Mon Jul 19, 2010 4:33 am

Thanks for the quick response. So then as I understand it, having 8gb on my machine, if I want to use all 4 processors, I'm limited to jobs which require less than 2gb for the largest array, correct?

JEJohns
Posts: 55
Joined: Sun May 02, 2010 5:30 pm

Re: Memory usage openmpi configuration

Post by JEJohns » Wed Jul 21, 2010 9:05 pm

bump

pmanglade
Posts: 20
Joined: Mon Aug 17, 2009 9:49 am

Re: Memory usage openmpi configuration

Post by pmanglade » Mon Jul 26, 2010 8:14 am

Yes in case your "jobs" meant processes : you send one job that splits into 4 processes ; each of those uses less than 2 GB of RAM.

Locked