Post-processing How-To

This is a list of FAQs about the usage of the AbiPy scripts. Feel free to suggest new entries!

Contents

• Post-processing How-To

  • Preliminary considerations

  • Get information about a generic FILE

  • Get all file extensions supported by abiopen.py

  • Convert the structure stored in FILE to a different format

  • Check if my Abinit input file is OK

  • Print the warnings in the log file

  • Get a quick look to a file

  • Visualize a structure

  • Get a high-symmetry kpath for a given structure

  • Re-symmetrize a structure when Abinit reports less symmetries than expected

  • Get neighbors for each atom in the unit cell out to a distance radius

  • Search on the Materials Project database for structures

  • Compare my structure with the Materials Project database

  • Visualize the iterations of the SCF cycle

  • Export bands to xmgrace format

  • Visualize the Fermi surface

  • Visualize phonon displacements

  • Visualize the results of a structural relaxation

  • Plot results stored in a text file in tabular format

  • Compare multiple files

  • Profile the scripts

  • Get the description of a variable

  • Avoid transfering files from the cluster to localhost just to use matplotlib

Preliminary considerations

The AbiPy scripts detect the file type by looking at the file extension so don’t change the file extension. Also, remember that it is possible to get the documentation by just adding the --help option to the command line. For example:

abistruct.py --help

gives the documentation and usage examples for the abistruct.py_ script while:

abistruct.py COMMAND --help

prints the documentation and the options supported by COMMAND

Get information about a generic FILE

Use:

abiopen.py FILE --print

to print information about a file inside the terminal or

abiopen.py FILE –expose

to generate multiple matplotlib figures depending on FILE.

Use --verbose or -v to increase verbosity level. The option can be can be supplied multiple times e.g. -vv.

Get all file extensions supported by abiopen.py

Use:

abiopen.py --help

$ abiopen.py --help
usage: abiopen.py [-h] [--loglevel LOGLEVEL] [-V] [-v] [-nb]
                  [--classic-notebook] [--no-browser] [--foreground] [-p]
                  [-pn] [-pnt PANEL_TEMPLATE] [-e] [-s] [-t SLIDE_TIMEOUT]
                  [-sns [SEABORN]] [-mpl MPL_BACKEND] [-ew] [-ply]
                  filepath

positional arguments:
  filepath              File to open. See table below for the list of
                        supported extensions.

optional arguments:
  -h, --help            show this help message and exit
  --loglevel LOGLEVEL   Set the loglevel. Possible values: CRITICAL, ERROR
                        (default), WARNING, INFO, DEBUG
  -V, --version         show program's version number and exit
  -v, --verbose         verbose, can be supplied multiple times to increase
                        verbosity
  -nb, --notebook       Open file in jupyter notebook
  --classic-notebook, -cnb
                        Use classic jupyter notebook instead of jupyterlab.
  --no-browser          Start the jupyter server to serve the notebook but
                        don't open the notebook in the browser. Use this
                        option to connect remotely from localhost to the
                        machine running the kernel
  --foreground          Run jupyter notebook in the foreground.
  -p, --print           Print python object and return.
  -pn, --panel          Open Dashboard in web browser, requires panel package.
  -pnt PANEL_TEMPLATE, --panel-template PANEL_TEMPLATE
                        Specify template for panel dasboard.Possible values
                        are: FastList, FastGrid, Golden, Bootstrap, Material,
                        React, Vanilla.Default: FastList
  -e, --expose          Open file and generate matplotlib figures
                        automatically by calling expose method.
  -s, --slide-mode      Iterate over figures. Expose all figures at once if
                        not given on the CLI.
  -t SLIDE_TIMEOUT, --slide-timeout SLIDE_TIMEOUT
                        Close figure after slide-timeout seconds (only if
                        slide-mode). Block if not specified.
  -sns [SEABORN], --seaborn [SEABORN]
                        Use seaborn settings. Accept value defining context in
                        ("paper", "notebook", "talk", "poster"). Default:
                        paper
  -mpl MPL_BACKEND, --mpl-backend MPL_BACKEND
                        Set matplotlib interactive backend. Possible values:
                        GTKAgg, GTK3Agg, GTK, GTKCairo, GTK3Cairo, WXAgg, WX,
                        TkAgg, Qt4Agg, Qt5Agg, macosx.See also:
                        https://matplotlib.org/faq/usage_faq.html#what-is-a-
                        backend.
  -ew, --expose-web     Generate matplotlib plots in $BROWSER instead of
                        X-server. WARNING: Not all the features are supported.
  -ply, --plotly        Generate plotly plots in $BROWSER instead of
                        matplotlib. WARNING: Not all the features are
                        supported.

======================================================================================================
Usage example:

    abiopen.py FILE          => Open file in ipython shell.
    abiopen.py FILE -p       => Print info on object to terminal.
    abiopen.py FILE -e       => Generate matplotlib figures automatically.
                                Use -sns to activate seaborn settings.
    abiopen.py FILE -eweb    => Generate matplotlib figures, show them in the $BROWSER.
    abiopen.py FILE -ply     => Generate plotly figures automatically. Show them in the $BROWSER.
                                Note that not all FILEs support plotly.
    abiopen.py FILE -pn      => Generate GUI in web BROWSER to interact with FILE
                                Requires panel package (WARNING: still under development!)
    abiopen.py FILE -nb      => Generate jupyter-lab notebook.
    abiopen.py FILE -cnb     => Generate classic jupyter notebook.

where `FILE` is any file supported by abipy/pymatgen e.g. Netcdf files, Abinit input, POSCAR, xsf.
File extensions supported (including zipped files with extension in ".bz2", ".gz", ".z"):
Use `-v` to increase verbosity level (can be supplied multiple times e.g -vv).

JSON file are supported as well. In this case, abiopen.py tries to reconstruct python objects
assuming JSON document in MSONable format and then invokes ipython with the `data` object.
Use `-e` or `--notebook` or `--panel` to print the JSON dictionary without reconstructing python objects.
======================================================================================================

Table mapping file extension to AbiPy object:

Extension          Class
-----------------  -----------------------------------------------------
.abi               <class 'abipy.abio.abivars.AbinitInputFile'>
.in                <class 'abipy.abio.abivars.AbinitInputFile'>
.abo               <class 'abipy.abio.outputs.AbinitOutputFile'>
.out               <class 'abipy.abio.outputs.AbinitOutputFile'>
.log               <class 'abipy.abio.outputs.AbinitLogFile'>
.cif               <class 'abipy.core.structure.Structure'>
.abivars           <class 'abipy.core.structure.Structure'>
.ucell             <class 'abipy.core.structure.Structure'>
POSCAR             <class 'abipy.core.structure.Structure'>
.cssr              <class 'abipy.core.structure.Structure'>
.cube              <class 'abipy.core.mixins.CubeFile'>
anaddb.nc          <class 'abipy.dfpt.anaddbnc.AnaddbNcFile'>
DEN                <class 'abipy.electrons.denpot.DensityFortranFile'>
.psp8              <class 'pymatgen.io.abinit.pseudos.Pseudo'>
.pspnc             <class 'pymatgen.io.abinit.pseudos.Pseudo'>
.fhi               <class 'pymatgen.io.abinit.pseudos.Pseudo'>
JTH.xml            <class 'pymatgen.io.abinit.pseudos.Pseudo'>
.wout              <class 'abipy.wannier90.wout.WoutFile'>
COHPCAR.lobster    <class 'abipy.electrons.lobster.CoxpFile'>
COOPCAR.lobster    <class 'abipy.electrons.lobster.CoxpFile'>
ICOHPLIST.lobster  <class 'abipy.electrons.lobster.ICoxpFile'>
DOSCAR.lobster     <class 'abipy.electrons.lobster.LobsterDoscarFile'>
EDOS               <class 'abipy.electrons.ebands.EdosFile'>
GSR.nc             <class 'abipy.electrons.gsr.GsrFile'>
ESKW.nc            <class 'abipy.electrons.eskw.EskwFile'>
DEN.nc             <class 'abipy.electrons.denpot.DensityNcFile'>
OUT.nc             <class 'abipy.abio.outputs.OutNcFile'>
VHA.nc             <class 'abipy.electrons.denpot.VhartreeNcFile'>
VXC.nc             <class 'abipy.electrons.denpot.VxcNcFile'>
VHXC.nc            <class 'abipy.electrons.denpot.VhxcNcFile'>
POT.nc             <class 'abipy.electrons.denpot.PotNcFile'>
WFK.nc             <class 'abipy.waves.wfkfile.WfkFile'>
HIST.nc            <class 'abipy.dynamics.hist.HistFile'>
PSPS.nc            <class 'abipy.electrons.psps.PspsFile'>
DDB                <class 'abipy.dfpt.ddb.DdbFile'>
PHBST.nc           <class 'abipy.dfpt.phonons.PhbstFile'>
PHDOS.nc           <class 'abipy.dfpt.phonons.PhdosFile'>
SCR.nc             <class 'abipy.electrons.scr.ScrFile'>
SIGRES.nc          <class 'abipy.electrons.gw.SigresFile'>
GRUNS.nc           <class 'abipy.dfpt.gruneisen.GrunsNcFile'>
MDF.nc             <class 'abipy.electrons.bse.MdfFile'>
FATBANDS.nc        <class 'abipy.electrons.fatbands.FatBandsFile'>
FOLD2BLOCH.nc      <class 'abipy.electrons.fold2bloch.Fold2BlochNcfile'>
CUT3DDENPOT.nc     <class 'abipy.electrons.denpot.Cut3dDenPotNcFile'>
OPTIC.nc           <class 'abipy.electrons.optic.OpticNcFile'>
A2F.nc             <class 'abipy.eph.a2f.A2fFile'>
SIGEPH.nc          <class 'abipy.eph.sigeph.SigEPhFile'>
TRANSPORT.nc       <class 'abipy.eph.transportfile.TransportFile'>
RTA.nc             <class 'abipy.eph.rta.RtaFile'>
V1SYM.nc           <class 'abipy.eph.v1sym.V1symFile'>
GKQ.nc             <class 'abipy.eph.gkq.GkqFile'>
V1QNU.nc           <class 'abipy.eph.v1qnu.V1qnuFile'>
V1QAVG.nc          <class 'abipy.eph.v1qavg.V1qAvgFile'>
ABIWAN.nc          <class 'abipy.wannier90.abiwan.AbiwanFile'>

Convert the structure stored in FILE to a different format

Use:

abistruct.py convert FILE

to read the structure from FILE and generate a CIF file (default behaviour).

The majority of the netcdf files produced by Abinit contain structural information so this command can be used with netcdf output files as well as Abinit input/output files and all the other formats supported by pymatgen e.g. POSCAR files. Other formats can be specified with the -f option. For example:

abistruct.py convert FILE -f abivars

gives the Abinit input variables while:

abistruct.py convert FILE -f xsf > out.xsf

exports the structure to the xsf format (xcrysden) and save it to file.

Use:

abistruct.py convert --help

to list the formats supported.

Check if my Abinit input file is OK

First of all, one can use:

abiopen.py ../abipy/data/refs/si_ebands/run.abi -p

to print the crystalline structure and find the space group with the spglib library.

If the structure looks good, use the abinp.py script with the validate command as in:

abinp.py validate run.abi

to validate the input file with Abinit (requires manager.yml and, obviously, Abinit).

The script provides other options to invoke Abinit to get space group information, the list of k-points in the IBZ. the list of atomic perturbations for phonons or the list of autoparal configurations. See abinp.py --help for further info.

Print the warnings in the log file

Use:

abiopen.py run.log -p

to get:

Events found in /Users/gmatteo/git_repos/abipy/abipy/examples/flows/develop/flow_from_files/w0/t0/run.log

[1] <AbinitWarning at m_nctk.F90:568>
    netcdf lib does not support MPI-IO and: NetCDF: Parallel operation on file opened for non-parallel access

[2] <AbinitWarning at m_nctk.F90:588>
    The netcdf library does not support parallel IO, see message above
    Abinit won't be able to produce files in parallel e.g. when paral_kgb==1 is used.
    Action: install a netcdf4+HDF5 library with MPI-IO support.

[3] <AbinitWarning at m_hdr.F90:4258>
    input kptrlatt= 0 0 0 0 0 0 0 0 0  /= disk file kptrlatt=8 0 0 0 8 0 0 0 8

[4] <AbinitWarning at m_hdr.F90:4261>
    input kptopt= -2  /= disk file kptopt= 1

num_errors: 0, num_warnings: 4, num_comments: 0, completed: True

A similar interface is also available via:

abiview.py log run.log

Get a quick look to a file

The abiview.py script is especially designed for this task. The syntax is abiview.py COMMAND FILE where COMMAND is either the Abinit file extension (without .nc, if any) or the AbiPy object we want to visualize.

To get a quick look at the DDB file, use:

abiview.py ddb out_DDB

This command invokes anaddb to compute phonon bands and DOS from the DDB and produces matplotlib plots.

If FILE contains electronic band energies, use e.g.:

abiview.py ebands out_GSR.nc

to plot the KS eigenvalues (the same command works for other files such as WFK.nc, DEN.nc etc.

Note that abiview.py uses a predefined logic to visualize the data. There are options to tune some parameters and/or export data in different formats but exposing the AbiPy API from the command line is not easy.

For a more flexible interface, we suggest to use:

abiopen.py FILE

to start an ipython shell that will allow you to interact with the python object directly.

If you have installed jupyter on your machine/cluster and you have a web browser, use:

abiopen.py FILE -nb

to generate automatically a predefined jupyter notebook associated to the file type.

Visualize a structure

The visualization of the structure is delegated to external graphical applications that must be installed on your machine. AbiPy will extract the structure from FILE, convert it to one of the formats supported by the graphical application and finally invoke the executable. If you have vesta installed in one of the standard locations of your machine, you can simply execute:

abistruct.py visualize FILE

inside the terminal. Other applications can be specified with the --application option. At present, AbiPy supports vesta, ovito, xcrysden, avogadro, and v_sim.

To visualize the crystalline structure inside the jupyter notebook, you may want to try the nbjsmol jupyter extension.

Get a high-symmetry kpath for a given structure

Use the kpath command and pass a FILE providing structural info:

abistruct.py kpath FILE

to generate a template with the input variables defining the k-path

# Abinit Structure
natom 2
ntypat 1
typat 1 1
znucl 14
xred
   0.0000000000    0.0000000000    0.0000000000
   0.2500000000    0.2500000000    0.2500000000
acell    1.0    1.0    1.0
rprim
   0.0000000000    5.1085000000    5.1085000000
   5.1085000000    0.0000000000    5.1085000000
   5.1085000000    5.1085000000    0.0000000000

# K-path in reduced coordinates:
# tolwfr 1e-20 iscf -2 getden ??
ndivsm 10
kptopt -11
kptbounds
   +0.00000  +0.00000  +0.00000 # $\Gamma$
   +0.50000  +0.00000  +0.50000 # X
   +0.50000  +0.25000  +0.75000 # W
   +0.37500  +0.37500  +0.75000 # K
   +0.00000  +0.00000  +0.00000 # $\Gamma$
   +0.50000  +0.50000  +0.50000 # L
   +0.62500  +0.25000  +0.62500 # U
   +0.50000  +0.25000  +0.75000 # W
   +0.50000  +0.50000  +0.50000 # L
   +0.37500  +0.37500  +0.75000 # K
   +0.62500  +0.25000  +0.62500 # U
   +0.50000  +0.00000  +0.50000 # X

Re-symmetrize a structure when Abinit reports less symmetries than expected

Crystalline structures saved in text format (e.g. CIF files downloaded from the Materials Project websites) may not have enough significant digits and Abinit may not find the same spacegroup as the one reported by the source as the default tolerance for symmetry detection in Abinit is tight (tolsym = 1e-8).

In this case, one can use the abispg option of abistruct.py to compute the spacegroup with Abinit and a tolerance larger that the default value:

abistruct.py abispg problematic.cif --tolsym=1e-6

Hopefully, the code will detect the correct spacegroup, will re-symmetrize the initial lattice vectors and atomic positions and print the new symmetrized structure to terminal.

Get neighbors for each atom in the unit cell out to a distance radius

If we are interested in the environment/nearest neighbours of the atoms in the unit cell, we can analyze the different coordinations with:

abistruct.py neighbors sic_relax_HIST.nc

Finding neighbors for each atom in the unit cell, out to a distance 2 [Angstrom]

[0] site PeriodicSite: C (0.0000, -0.0000, 0.0000) [-0.0000, 0.0000, -0.0000] has 4 neighbors:
         PeriodicSite: Si (1.0836, -1.0836, -1.0836) [-0.7500, 0.2500, 0.2500]  at distance 1.8767766107
         PeriodicSite: Si (-1.0836, 1.0836, -1.0836) [0.2500, -0.7500, 0.2500]  at distance 1.8767766107
         PeriodicSite: Si (-1.0836, -1.0836, 1.0836) [0.2500, 0.2500, -0.7500]  at distance 1.8767766107
         PeriodicSite: Si (1.0836, 1.0836, 1.0836) [0.2500, 0.2500, 0.2500]  at distance 1.8767766107

[1] site PeriodicSite: Si (1.0836, 1.0836, 1.0836) [0.2500, 0.2500, 0.2500] has 4 neighbors:
         PeriodicSite: C (0.0000, 0.0000, 0.0000) [0.0000, 0.0000, 0.0000]  at distance 1.8767766107
         PeriodicSite: C (2.1671, 2.1671, 0.0000) [0.0000, 0.0000, 1.0000]  at distance 1.8767766107
         PeriodicSite: C (2.1671, 0.0000, 2.1671) [0.0000, 1.0000, 0.0000]  at distance 1.8767766107
         PeriodicSite: C (0.0000, 2.1671, 2.1671) [1.0000, 0.0000, 0.0000]  at distance 1.8767766107

Search on the Materials Project database for structures

Use:

abistruct.py mp_search LiF

to search on the materials project database for structures corresponding to a chemical system or formula e.g. Fe2O3 or Li-Fe-O or Ir-O-* for wildcard pattern matching.

The script prints the results to terminal in tabular form:

# Found 2 structures in materials project database (use `verbose` to get full info)
           pretty_formula  e_above_hull  energy_per_atom  \
mp-1138               LiF      0.000000        -4.845142
mp-1009009            LiF      0.273111        -4.572031

            formation_energy_per_atom  nsites     volume spacegroup.symbol  \
mp-1138                     -3.180439       2  17.022154             Fm-3m
mp-1009009                  -2.907328       2  16.768040             Pm-3m

            spacegroup.number  band_gap  total_magnetization material_id
mp-1138                   225    8.7161                  0.0     mp-1138
mp-1009009                221    7.5046                 -0.0  mp-1009009

Important

The script will try to connect to the materials project server. You need a ~/.pmgrc.yaml configuration file inside your home directory with the authentication token PMG_MAPI_KEY. For further info please refer to the pymatgen documentation

The script provides other commands to get (experimental) structures from the COD database, find matching structures on the materials project website and generate phase diagrams. See abistruct.py --help for more examples.

Compare my structure with the Materials Project database

Let’s assume we have performed a structural relaxation and we want to compare our results with the Materials Project data. One can use the abicomp.py structure to extract the structure from the HIST.nc file and compare the data with the database:

abicomp.py mp_structure ../abipy/data/refs/sic_relax_HIST.nc

It’s also possible to select only the structures with the same space group number as the input structure with:

abicomp.py mp_structure ../abipy/data/refs/sic_relax_HIST.nc --same-spgnum

that produces

Lattice parameters:
        formula  natom  angle0  angle1  angle2         a         b         c  \
mp-8062  Si1 C1      2    60.0    60.0    60.0  3.096817  3.096817  3.096817
this     Si1 C1      2    60.0    60.0    60.0  3.064763  3.064763  3.064763

            volume abispg_num spglib_symb  spglib_num
mp-8062  21.000596       None       F-43m         216
this     20.355222       None       F-43m         216

Use --verbose to print atomic positions.

Note that one can replace the HIST.nc file with any other file providing a structure object.

Important

The structures of the materials project have been obtained with the GGA-PBE functional and they might include the U term in the Hamiltonian. One should take into account these different settings when comparing structural relaxations.

Visualize the iterations of the SCF cycle

Use:

abiview.py abo run.abo

to plot the SCF iterations or the steps of the structural relaxations or the DFPT SCF cycles (depending on the content of run.abo).

Note that one can also use:

abiview.py log run.log

to print the warnings/comments/errors reported in the Abinit log file run.log.

Export bands to xmgrace format

Both abipy.electrons.ebands.ElectronBands and abipy.dfpt.phonons.PhononBands provide a to_xmgrace method to produce xmgrace files. To export the data to xmgrace, use abiview.py with the --xmgrace option. For electrons, use:

abiview.py ebands out_GSR.nc --xmgrace

and:

abiview.py phbands out_PHBST.nc -xmgrace

for phonons.

Visualize the Fermi surface

Use:

abiview.py ebands out_GSR.nc --bxsf

to export a set of band energies in BXSF format suitable for the visualization of the Fermi surface with xcrysden. Then use:

xcrysden --bxsf BXSF_FILE

to visualize the Fermi surface with xcrysden

abifile.ebands.to_bxsf("mgb2.bxsf")

Important

This option requires k-points in the irreducible wedge and a gamma-centered k-mesh.

Visualize phonon displacements

AbiPy is interfaced with the phononwebsite project If you have already installed the python package from github it’s possbile to export the PHBST.nc to JSON and then load the file via the web-interface. Alternatively, it’s possible to automate the entire procedure with the abiview.py script.

Use:

abiview.py phbands out_PHBST.nc -web

to start a local web server and open the HTML page inside the default browser (the browser can be changed with the --browser option).

It is also possible to visualize the phonon modes starting directly from a DDB file with:

abiview.py ddb -web

In this case, AbiPy will invoke anaddb to produce the PHBST.nc file on an automatically generated q-path and then start the web server.

Visualize the results of a structural relaxation

The quickest way is to use:

abiview hist out_HIST.nc

to plot the results with matplotlib or:

abiopen.py out_HIST.nc -p

to print the most important results to terminal.

Note that it’s possible to generate a XDATCAR file with:

abiview hist out_HIST.nc --xdatcar

and visualize the crystalline structure with ovito:

abiview hist out_HIST.nc -appname=ovito

Important

The XDATCAR format assumes a fixed unit cell so you won’t be able to visualize the modifications of the unit cell lattice vectors in ovito.

Plot results stored in a text file in tabular format

Use:

abiview.py data FILE_WITH_COLUMNS

to plot with matplotlib all the columns given in the file. By default, the first column is assumed to contain the values for the x-axis but it is possible to change this behaviour and use the row index with the –use-index option. Multiple datasets i.e. multiple sets of data separated by blank lines are supported.

To compare multiple files use:

abicomp.py data FILE1 FILE2

Obviously one can use standard tools such as gnuplot and xmgrace but the AbiPy scripts are quite handy for a quick analysis of the results.

Compare multiple files

The abicomp.py script is explicitly designed for this kind of task. It operates on multiple files (usually files with the same extension) and either produces matplotlib plots or creates AbiPy robots providing methods to analyze the results, perform convergence studies and build pandas DataFrames.

The COMMAND defines the quantity to be compared, followed by a list of filenames.

To compare e.g. the structure given in one Abinit input file with the structure coming from a GSR.nc file, use:

abicomp.py structure run.abi out_GSR.nc

Note

In this example, we can use files of different type because they both have a Structure object. This philosophy can be applied to other commands as well: everything works as long as AbiPy is able to extract the quantity of interest from the file.

To plot multiple electronic structures on a grid, use:

abicomp.py ebands *_GSR.nc out2_WFK.nc -p

Remember that it is possible to use the shell syntax *_GSR.nc to select all files with a given extension. If you have nested directories, use unix find to scan the directory tree for files matching a given pattern For example:

abicomp.py ebands `find . -name *_GSR.nc`

finds all GSR.nc files contained withing the current working directory. The output of find is then passed to the abicomp.py script.

Note

Note the backticks syntax used in the command.

Profile the scripts

All AbiPy script can be executed in profile mode by just prepending the prof keyword to the command line arguments. This option could be useful if the script seems to be slow and you need to understand what’s happening.

Use:

abiopen.py prof FILE

or:

abistruct.py prof COMMAND FILE

if the script requires a COMMAND argument.

Get the description of a variable

The abidoc.py script provides a simplified interface to the Abinit documentation.

Use:

abidoc.py man ecut

to print the official documentation for the ecut variable to the terminal.

To list all the variables depending on the natom dimension, use:

abidoc.py withdim natom

More options are available. See abidoc.py --help.

Avoid transfering files from the cluster to localhost just to use matplotlib

Use SSHFS to mount the remote file system over SSH. Now one can execute the AbiPy scripts in a terminal running on the local machine. to open/visualize the files stored on the cluster.