job termination: lack of memory. 32bit vs. 64bit problem?

option, parallelism,...

Moderators: fgoudreault, mcote

Forum rules
Please have a look at ~abinit/doc/config/build-config.ac in the source package for detailed and up-to-date information about the configuration of Abinit 8 builds.
For a video explanation on how to build Abinit 7.x for Linux, please go to: http://www.youtube.com/watch?v=DppLQ-KQA68.
IMPORTANT: when an answer solves your problem, please check the little green V-like button on its upper-right corner to accept it.
Locked
spamrefuse
Posts: 35
Joined: Wed Jan 20, 2010 3:08 am

job termination: lack of memory. 32bit vs. 64bit problem?

Post by spamrefuse » Tue Sep 28, 2010 3:00 pm

Hi,

Abinit 6.2.3 runs on a up-to-date Fedora system
(32 bits and with kernel 2.6.32-PAE).
The system has 6 GB RAM and 12 GB swap.

When I run the abinit job, it terminates because it cannot allocate
2779.477 Mb (see below for snippet from the log file).

How come this job cannot find 2779.477 MB (= 2.7 GB) ?

The 'top' command tells me that 90 % of the swap is not used
and 2 GB of RAM is free.

Am I facing here the limitation of a 32 bits version of the Linux OS?
If I replace the 32 bits OS with its equivalent 64 bits version,
would that help?

My PC has an Intel Core i7 CPU, which effectively shows as 8 CPUs.
Would parallelization with mpi on the localhost do any good to solve
this memory problem?

Thank you!
Rob.


Here is the snippet of the log file:

[...snip...]
=========================================================
P This job should need less than 2779.477 Mbytes of memory.
Rough estimation (10% accuracy) of disk space for files :
WF disk file : -1573.915 Mbytes ; DEN or POT disk file : 9.613 Mbytes.
=========================================================

Biggest array : f_fftgr(disk), with 9.6128 MBytes.
memana : allocated an array of 9.613 Mbytes, for testing purposes.

memana : ERROR -
Test failed to allocate 2779.477 Mbytes
It is not worth to continue
Action : modify input variable to fit the available memory.
or increase limit on available memory.

leave_new : decision taken to exit ...
=========================================================

User avatar
pouillon
Posts: 651
Joined: Wed Aug 19, 2009 10:08 am
Location: Spain
Contact:

Re: job termination: lack of memory. 32bit vs. 64bit problem

Post by pouillon » Thu Sep 30, 2010 12:10 pm

You should first be aware that Abinit tends to underestimate memory requirements.

With a 32-bits OS, you can use up to 3Gb of memory. Going to 64 bits is interesting only if you have more physical memory.

In your case, using MPI will not help, since the resources available will not increase. You will probably need even more memory if you do so.
Yann Pouillon
Simune Atomistics
Donostia-San Sebastián, Spain

mverstra
Posts: 655
Joined: Wed Aug 19, 2009 12:01 pm

Re: job termination: lack of memory. 32bit vs. 64bit problem

Post by mverstra » Mon Oct 11, 2010 11:39 am

Hard to tell if the memory limitation is real or enforced by the 32 bit stuff. You can upgrade your system and recompile abinit as a 64 bit executable. Running in mpi might actually help if each of the threads is using less memory (can be handled by 32 bits) even though the total will be more (but still less than your 6 Gb).

Cheers

Matthieu
Matthieu Verstraete
University of Liege, Belgium

User avatar
pouillon
Posts: 651
Joined: Wed Aug 19, 2009 10:08 am
Location: Spain
Contact:

Re: job termination: lack of memory. 32bit vs. 64bit problem

Post by pouillon » Fri Oct 15, 2010 11:26 am

In any case, with a 32-bit Linux, the kernel will only be able to address 3Gb of physical memory at most. It is really worth going to 64-bit when you have more memory.
Yann Pouillon
Simune Atomistics
Donostia-San Sebastián, Spain

Locked