[SOLVED] Problem about configure 6.6.1 on Blue Gene

xiangpisai · Post by **xiangpisai** » Thu Mar 24, 2011 3:10 pm

Hi all,
I tried to run the configure file on Blue Gene but everytime it gives me an error. My hostname.ac file is as follows:

CC="mpixlc"
CXX="mpixlcxx"
FC="mpixlf90"
F77="mpixlf77"
enable_fc_wrapper="yes"
#enable_stdin="no"
enable_mpi="yes"
enable_mpi_io="yes"
MPI_RUNNER="/bgl/BlueLight/ppcfloor/bglsys/bin/mpirun"
#enable_dft="no"
#with_dft_flavor="libxc"
#enable_netcdf="no"
#enable_wannier90="no"

When I compile, I use cross-compiling by specifying --host=powerpc-unknown-linux. However this prompts an error saying

checking whether to build netcdf... yes
checking whether to build wannier90... yes
configure: using tarball repository /XXXX(my directory)/.abinit/tarballs
checking for a source tarball of LINALG... no
configure: downloading LINALG - this may take a while
checking availability of LINALG from URL 1... no
configure: WARNING: could not download LINALG fallback tarball
configure: error: cannot build Abinit - aborting

But if I tried to uncomment the last line of my hostname.ac file, it will not run at all.
Can anybody give me a hint about this error? Thank you so much!

Boris · Post by **Boris** » Thu Mar 24, 2011 3:43 pm

Hi

Abinit stops because it cannot find the linear algebra libraries. Therefore, you need to specify yourself the linear algebra libraries that you want to use. I think mkl is available on Blue Gene. So try to to add (for instance with mkl):

Code: Select all

 --with-linalg-flavor=mkl  --with-linalg-libs="-L/path/to/mkl/libraries/lib/em64 -lblas -llapack"

Boris

Edit : Or maybe you can download the lapack libraries on the abinit website, put them in the /tarballs directory and Abinit will install them automatically. I did not try this option but it should work.

Alain_Jacques · Post by **Alain_Jacques** » Thu Mar 24, 2011 4:03 pm

When Abinit configure doesn't find suitable Lapack/Blas libraries, it tries to download a plain distribution from Abinit website using wget. For a reason I cannot guess without config.log, configure fails to get it - maybe because your BG system doesn't allow to access Internet for security reasons. The packages are available from http://www.abinit.org/downloads/plug-in-sources. Use whatever method to obtain and copy them to /XXXX(my directory)/.abinit/tarballs/ and configure will find them there. Caution: the atompaw package is outdated, get http://archives.abinit.org/plugins/atompaw-3.0.1.tar.gz instead

Kind regards,

Alain

Alain_Jacques · Post by **Alain_Jacques** » Thu Mar 24, 2011 4:10 pm

... as far as I know, Blue Gene is PowerPC based - I would be rather surprised if Intel provides optimized libraries to competitors. On PowerPC architecture, the equivalent of MKL is ESSL but this library hasn't been recently tested with Abinit.

Alain

Boris · Post by **Boris** » Thu Mar 24, 2011 4:15 pm

Alain_Jacques wrote:... as far as I know, Blue Gene are PowerPC based - I would be rather surprised if Intel provides optimized libraries to competitors. On PowerPC architecture, the equivalent of MKL is ESSL but this library hasn't been tested with MKL.

Alain

Oh, ok. Good to know. Then I guess the simplest way of doing this is to DL the lapack libraries on the abinit website.

xiangpisai · Post by **xiangpisai** » Thu Mar 24, 2011 4:44 pm

Alain_Jacques wrote:When Abinit configure doesn't find suitable Lapack/Blas libraries, it tries to download a plain distribution from Abinit website using wget. For a reason I cannot guess without config.log, configure fails to get it - maybe because your BG system doesn't allow to access Internet for security reasons. The packages are available from http://www.abinit.org/downloads/plug-in-sources. Use whatever method to obtain and copy them to /XXXX(my directory)/.abinit/tarballs/ and configure will find them there. Caution: the atompaw package is outdated, get http://archives.abinit.org/plugins/atompaw-3.0.1.tar.gz instead

Kind regards,

Alain

Hi, I used your method and the configure process went out smoothly. Thank you very much! But when I type "make" command, it gave me an error after 5 minutes or so, saying "cannot run C compiled files, if you mean to cross-compile, use --host". I have met this error many times before because I am using cross-compiling. In my configure command, I specified --host=powerpc-unknown-linux, which is correct. But I don't think make --host=powerpc-unknown-linux will work since this is not an option of make.
Could you please tell me where the problem is? Thank you very much!
Best regards

BTW: I have attached my config.log file, after the make command. You may check it if needed. This is actually a .zip file. Since the original file is over 256kb size and the forum won't accept the compressed format(zip), I have to compress it and modify the extension. So please change the extension to zip and extract it. Thank you very much!

More to mention:
There seems no change after the make command, so I am putting the result of the make command:

Code: Select all

checking whether a win32 DLL is desired... no
checking whether separate fortran libs are desired... no
configure: finding C compiler
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... configure: error: in `/XXXX/abinit-6.6.1/plugins/netcdf/netcdf-4.1.1':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details.
make[4]: *** [configure-stamp] Error 1
make[4]: Leaving directory `/XXXX/abinit-6.6.1/plugins/netcdf'
make[3]: *** [package-ready] Error 2
make[3]: Leaving directory `/XXXX/abinit-6.6.1/plugins/netcdf'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/XXXX/abinit-6.6.1/plugins'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/XXXX/abinit-6.6.1'
make: *** [all] Error 2

Post by **pouillon** » Thu Mar 24, 2011 5:14 pm

Without more precise information, I cannot pinpoint the exact problem you're encountering.

However, this is likely an error during the compilation of an external package. If you don't need any advanced feature, you can try to configure with the following options:

Code: Select all

../configure --with-trio-flavor="none" --with-dft-flavor="none"

This will remove support for NetCDF, ETSF_IO, FoX, AtomPAW, BigDFT, LibXC, and Wannier90.

You may also try with only one of these two options.

xiangpisai · Post by **xiangpisai** » Fri Mar 25, 2011 5:02 am

pouillon wrote:Without more precise information, I cannot pinpoint the exact problem you're encountering.

However, this is likely an error during the compilation of an external package. If you don't need any advanced feature, you can try to configure with the following options:
Code: Select all
../configure --with-trio-flavor="none" --with-dft-flavor="none"
This will remove support for NetCDF, ETSF_IO, FoX, AtomPAW, BigDFT, LibXC, and Wannier90.

You may also try with only one of these two options.

Hi, I used your option and it worked well! Thank you very much! But there is a new problem. I run the tgw1_1.in file with bluegene. I specified 512 nodes to it and it takes 180 seconds to complete the calculation. But if I use 32 AMD processors it takes only 43 seconds. In our common sense, Blue Gene is faster than normal AMD processors. But why there is such a big difference? I don't know what information you guys need, so would you please reply to tell me that? Thank you so much!
Best regards!
xiangpisai

Alain_Jacques · Post by **Alain_Jacques** » Fri Mar 25, 2011 11:12 am

Dear xiangpisai,

Despite the common sense proverb saying quod abundat non vitiat, it is quite easy to do less with more.

First of all, the input files of the test suite are designed to check accuracy, not as benchmarks for speed - at least not without modifications. So don't take any of them, launch it on 512 cpus and expect a 512 fold increase of performance. Abinit has several options to parallelize computations, some of them scale linealy, others not - the art of clusters is to weight resources vs. efficiency, not to use brute force.

For the particular case of tgw1_1, considering the plain k-points grid as defined, I would say that about a quarter of your 512 cpus were busy calculating and the rest slowing down the process (and it's probably even worse than that).
So please have a look on the tutorials and variables relevant to Abinit parallelization and, in particular, the paral_kgb variable with negative values that can provide a guidance about the efficiency of parallel options in your specific input. And maybe look at tests/paral/Input/si_kpt_band_fft.in as a starting point to design a large scale benchmark and study the effects of the different combinations.

Kind regards,

Alain

xiangpisai · Post by **xiangpisai** » Fri Mar 25, 2011 6:52 pm

Alain_Jacques wrote:Dear xiangpisai,

Despite the common sense proverb saying quod abundat non vitiat, it is quite easy to do less with more.

First of all, the input files of the test suite are designed to check accuracy, not as benchmarks for speed - at least not without modifications. So don't take any of them, launch it on 512 cpus and expect a 512 fold increase of performance. Abinit has several options to parallelize computations, some of them scale linealy, others not - the art of clusters is to weight resources vs. efficiency, not to use brute force.

For the particular case of tgw1_1, considering the plain k-points grid as defined, I would say that about a quarter of your 512 cpus were busy calculating and the rest slowing down the process (and it's probably even worse than that).
So please have a look on the tutorials and variables relevant to Abinit parallelization and, in particular, the paral_kgb variable with negative values that can provide a guidance about the efficiency of parallel options in your specific input. And maybe look at tests/paral/Input/si_kpt_band_fft.in as a starting point to design a large scale benchmark and study the effects of the different combinations.

Kind regards,

Alain

Dear Alain,
I will do it. Thank you so much for your kind help.
Best regards
Xiangpisai

ABINIT Discussion Forums

[SOLVED] Problem about configure 6.6.1 on Blue Gene

[SOLVED] Problem about configure 6.6.1 on Blue Gene

Re: Problem about configure 6.6.1 on Blue Gene

Re: Problem about configure 6.6.1 on Blue Gene

Re: Problem about configure 6.6.1 on Blue Gene

Re: Problem about configure 6.6.1 on Blue Gene

Re: Problem about configure 6.6.1 on Blue Gene

Re: Problem about configure 6.6.1 on Blue Gene

Re: Problem about configure 6.6.1 on Blue Gene

Re: Problem about configure 6.6.1 on Blue Gene

Re: Problem about configure 6.6.1 on Blue Gene