flowtk Package

Contents

• flowtk Package

  • flowtk Package

  • abiinspect Module

  • abiobjects Module

  • abiphonopy Module

  • abitimer Module

  • dfpt_works Module

  • effmass_works Module

  • eph_flows Module

  • events Module

  • flows Module

  • gw Module

  • gruneisen Module

  • gs_works Module

  • launcher Module

  • lumi_works Module

  • mocks Module

  • netcdf Module

  • nodes Module

  • pseudos Module

  • qadapters Module

  • qha Module

  • qjobs Module

  • qutils Module

  • tasks Module

  • utils Module

  • works Module

  • wrappers Module

flowtk Package

abipy.flowtk.flow_main(main)

This decorator is used to decorate main functions producing Flows. It adds the initialization of the logger and an argument parser that allows one to select the loglevel, the workdir of the flow as well as the YAML file with the parameters of the TaskManager. The main function shall have the signature:

main(options)

where options in the container with the command line options generated by ArgumentParser.

Parameters

main – main function.

abipy.flowtk.build_flow_main_parser()

Build and return the parser used in the abipy/data/runs scripts.

abiinspect Module

abiobjects Module

class abipy.flowtk.abiobjects.LujForSpecie(l, u, j, unit)

Bases: abipy.flowtk.abiobjects.LdauForSpecie

This object stores the value of l, u, j used for a single atomic specie.

class abipy.flowtk.abiobjects.LdauParams(usepawu, structure)

Bases: object

This object stores the parameters for LDA+U calculations with the PAW method It facilitates the specification of the U-J parameters in the Abinit input file. (see to_abivars). The U-J operator will be applied only on the atomic species that have been selected by calling lui_for_symbol.

To setup the Abinit variables for a LDA+U calculation in NiO with a U value of 5 eV applied on the nickel atoms:

luj_params = LdauParams(usepawu=1, structure=nio_structure)
# Apply U-J on Ni only.
u = 5.0
luj_params.luj_for_symbol("Ni", l=2, u=u, j=0.1*u, unit="eV")

print(luj_params.to_abivars())

property symbols_by_typat

luj_for_symbol(symbol, l, u, j, unit='eV')

Parameters

• symbol – Chemical symbol of the atoms on which LDA+U should be applied.

• l – Angular momentum.

• u – Value of U.

• j – Value of J.

• unit – Energy unit of U and J.

to_abivars()

Returns a dict with the Abinit variables.

class abipy.flowtk.abiobjects.LexxParams(structure)

Bases: object

This object stores the parameters for local exact exchange calculations with the PAW method It facilitates the specification of the LEXX parameters in the Abinit input file. (see to_abivars). The LEXX operator will be applied only on the atomic species that have been selected by calling lexx_for_symbol.

To perform a LEXX calculation for NiO in which the LEXX is computed only for the l=2 channel of the nickel atoms:

lexx_params = LexxParams(nio_structure)
lexx_params.lexx_for_symbol("Ni", l=2)

print(lexc_params.to_abivars())

property symbols_by_typat

lexx_for_symbol(symbol, l)

Enable LEXX for the given chemical symbol and the angular momentum l

Parameters

• symbol – Chemical symbol of the atoms on which LEXX should be applied.

• l – Angular momentum.

to_abivars()

Returns a dict with the Abinit variables.

abiphonopy Module

Interface between phonopy and abipy workflow model.

abipy.flowtk.abiphonopy.atoms_from_structure(structure)

Convert a pymatgen Structure into a phonopy Atoms object.

abipy.flowtk.abiphonopy.structure_from_atoms(atoms)

Convert a phonopy Atoms object into a pymatgen Structure.

class abipy.flowtk.abiphonopy.PhonopyWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

This work computes the inter-atomic force constants with phonopy.

scdims(3)

numpy arrays with the number of cells in the supercell along the three reduced directions

phonon

Phonopy object used to construct the supercells with displaced atoms.

phonopy_tasks

List of ScfTask. Each task compute the forces in one perturbed supercell.

bec_tasks

cpdata2dst

If not None, the work will copy the output results to the outdir of the flow once all_ok is invoked. Note that cpdata2dst must be an absolute path.

classmethod from_gs_input(gsinp, scdims, phonopy_kwargs=None, displ_kwargs=None)

Build the work from an AbinitInput object representing a GS calculations.

Parameters

• gsinp – AbinitInput object representing a GS calculation in the initial unit cell.

• scdims – Number of unit cell replicas along the three reduced directions.

• phonopy_kwargs – (Optional) dictionary with arguments passed to Phonopy constructor.

• displ_kwargs – (Optional) dictionary with arguments passed to generate_displacements.

Return: PhonopyWork instance.

on_all_ok()

This method is called once the Work is completed i.e. when all the tasks have reached status S_OK. Here we get the forces from the output files, and we call phonopy to compute the inter-atomic force constants.

class abipy.flowtk.abiphonopy.PhonopyGruneisenWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

This work computes the Grüneisen parameters with phonopy. The workflow is as follows:

It is necessary to run three phonon calculations. One is calculated at the equilibrium volume and the remaining two are calculated at the slightly larger volume and smaller volume than the equilibrium volume. The unitcells at these volumes have to be fully relaxed under the constraint of each volume.

scdims(3)

numpy arrays with the number of cells in the supercell along the three reduced directions.

classmethod from_gs_input(gsinp, voldelta, scdims, phonopy_kwargs=None, displ_kwargs=None)

Build the work from an AbinitInput object representing a GS calculations.

Parameters

• gsinp – AbinitInput object representing a GS calculation in the initial unit cell.

• voldelta – Absolute increment for unit cell volume. The three volumes are: [v0 - voldelta, v0, v0 + voldelta] where v0 is taken from gsinp.structure.

• scdims – Number of unit cell replicas along the three reduced directions.

• phonopy_kwargs – (Optional) dictionary with arguments passed to Phonopy constructor.

• displ_kwargs – (Optional) dictionary with arguments passed to generate_displacements.

Return: PhonopyGruneisenWork instance.

on_all_ok()

This method is called once the Work is completed i.e. when all the tasks have reached status S_OK.

add_phonopy_works_and_build()

Get relaxed structures from the tasks, build Phonopy works with supercells constructed from the new structures, add them to the flow and build new directories.

abitimer Module

dfpt_works Module

Work subclasses related to DFTP.

class abipy.flowtk.dfpt_works.ElasticWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work, abipy.flowtk.works.MergeDdb

This Work computes the elastic constants and (optionally) the piezoelectric tensor. It consists of Response function calculations for:

• rigid-atom elastic tensor

• rigid-atom piezoelectric tensor

• interatomic force constants at gamma

• Born effective charges

The structure is assumed to be already relaxed

Create a Flow for phonon calculations. The flow has one works with:

• 1 GS Task

• 3 DDK Task

• 4 Phonon Tasks (Gamma point)

• 6 Elastic tasks (3 uniaxial + 3 shear strain)

The Phonon tasks and the elastic task will read the DDK produced at the beginning

classmethod from_scf_input(scf_input, with_relaxed_ion=True, with_piezo=False, with_dde=False, tolerances=None, den_deps=None, manager=None)

Parameters

• scf_input –

• with_relaxed_ion –

• with_piezo –

• with_dde – Compute electric field perturbations.

• tolerances – Dict of tolerances

• den_deps –

• manager –

Similar to from_scf_task, the difference is that this method requires an input for SCF calculation instead of a ScfTask. All the tasks (Scf + Phonon) are packed in a single Work whereas in the previous case we usually have multiple works.

on_all_ok()

This method is called when all the tasks of the Work reach S_OK. Ir runs mrgddb in sequential on the local machine to produce the final DDB file in the outdir of the Work.

class abipy.flowtk.dfpt_works.NscfDdksWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

This work requires a DEN file and computes the KS energies with a non self-consistent task with a dense k-mesh and empty states. This task is then followed by the computation of the DDK matrix elements with nstep = 1 (the first order change of the wavefunctions is not converged but we only need the matrix elements) Mainly used to prepare optic calculations or other post-processing steps requiring the DDKs.

classmethod from_scf_task(scf_task, ddk_ngkpt, ddk_shiftk, ddk_nband, manager=None)

Build NscfDdksWork from a scf_task.

Parameters

• scf_task – GS task. Must produce the DEN file required for the NSCF run.

• ddk_ngkpt – k-mesh used for the NSCF run and the non self-consistent DDK tasks.

• ddk_shiftk – k-mesh shifts

• ddk_nband – Number of bands (occupied + empty) used in the NSCF task and the DDKs tasks.

• manager – TaskManager instance. Use default if None.

Return: NscfDdksWork instance

effmass_works Module

Work subclasses related to effective mass calculations.

class abipy.flowtk.effmass_works.EffMassLineWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

Work for the computation of effective masses via finite differences along a k-line. Useful for cases such as NC+SOC where DFPT is not implemented or if one is interested in non-parabolic behaviour.

Inheritance Diagram

classmethod from_scf_input(scf_input, k0_list, step=0.01, npts=15, red_dirs=[[1, 0, 0], [0, 1, 0], [0, 0, 1]], cart_dirs=None, den_node=None, manager=None)

Build the Work from an abipy.abio.inputs.AbinitInput representing a GS-SCF calculation.

Parameters

• scf_input – abipy.abio.inputs.AbinitInput for GS-SCF used as template to generate the other inputs.

• k0_list – List with the reduced coordinates of the k-points where effective masses are wanted.

• step – Step for finite difference in Angstrom^-1

• npts – Number of points sampled around each k-point for each direction.

• red_dirs – List of reduced directions used to generate the segments passing through the k-point

• cart_dirs – List of Cartesian directions used to generate the segments passing through the k-point

• den_node – Path to the DEN file or Task object producing a DEN file. Can be used to avoid the initial SCF calculation if a DEN file is already available. If None, a GS calculation is performed.

• manager – abipy.flowtk.tasks.TaskManager instance. Use default if None.

class abipy.flowtk.effmass_works.EffMassDFPTWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

Work for the computation of effective masses with DFPT. Requires explicit list of k-points and range of bands.

Inheritance Diagram

classmethod from_scf_input(scf_input, k0_list, effmass_bands_f90, ngfft=None, den_node=None, manager=None)

Build the Work from an abipy.abio.inputs.AbinitInput representing a GS-SCF calculation.

Parameters

• scf_input – abipy.abio.inputs.AbinitInput for GS-SCF used as template to generate the other inputs.

• k0_list – List with the reduced coordinates of the k-points where effective masses are wanted.

• effmass_bands_f90 – (nkpt, 2) array with band range for effmas computation. WARNING: Assumes Fortran convention with indices starting from 1.

• ngfft – FFT divisions (3 integers). Used to enforce the same FFT mesh in the NSCF run as the one used for GS.

• den_node – Path to the DEN file or Task object producing a DEN file. Can be used to avoid the initial SCF calculation if a DEN file is already available. If None, a GS calculation is performed.

• manager – abipy.flowtk.tasks.TaskManager instance. Use default if None.

class abipy.flowtk.effmass_works.EffMassAutoDFPTWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

Work for the automatic computation of effective masses with DFPT. Band extrema are automatically detected by performing a NSCF calculation along a high-symmetry k-path with ndivsm. Requires more computation that EffMassWork since input variables (kpoints and band range) are computed at runtime.

Inheritance Diagram

classmethod from_scf_input(scf_input, ndivsm=15, tolwfr=1e-20, den_node=None, manager=None)

Build the Work from an abipy.abio.inputs.AbinitInput representing a GS-SCF calculation.

Parameters

• scf_input – abipy.abio.inputs.AbinitInput for GS-SCF used as template to generate the other inputs.

• ndivsm – if > 0, it’s the number of divisions for the smallest segment of the path (Abinit variable). if < 0, it’s interpreted as the pymatgen line_density parameter in which the number of points in the segment is proportional to its length. Typical value: -20. This option is the recommended one if the k-path contains two high symmetry k-points that are very close as ndivsm > 0 may produce a very large number of wavevectors.

• tolwfr – Tolerance on residuals for NSCF calculation

• den_node – Path to the DEN file or Task object producing a DEN file. Can be used to avoid the initial SCF calculation if a DEN file is already available. If None, a GS calculation is performed.

• manager – abipy.flowtk.tasks.TaskManager instance. Use default if None.

on_all_ok()

This method is called once the Work is completed i.e. when all tasks have reached status S_OK. Here, we read the band structure from GSR to find the position of the band edges and use these values to generate a new work for effective masses with DFPT.

class abipy.flowtk.effmass_works.FrohlichZPRFlow(workdir, manager=None, pickle_protocol=- 1, remove=False)

Bases: abipy.flowtk.flows.Flow

Inheritance Diagram

classmethod from_scf_input(workdir, scf_input, ddb_node=None, ndivsm=15, tolwfr=1e-20, metadata=None, manager=None)

Build the Work from an abipy.abio.inputs.AbinitInput representing a GS-SCF calculation. Final results are stored in the “zprfrohl_results.json” in the outdata directory of the flow.

Parameters

• workdir – Working directory.

• scf_input – abipy.abio.inputs.AbinitInput for GS-SCF used as template to generate the other inputs.

• ddb_node – Path to an external DDB file that is used to avoid the calculation of BECS/eps_inf and phonons. If None, a DFPT calculation is automatically performed by the flow.

• ndivsm – Number of divisions used to sample the smallest segment of the k-path.

• tolwfr – Tolerance on residuals for NSCF calculation

• manager – abipy.flowtk.tasks.TaskManager instance. Use default if None.

• metadata – Dictionary with metadata addeded to the final JSON file.

on_all_ok()

This method is called when all the works in the flow have reached S_OK. This method shall return True if the calculation is completed or False if the execution should continue due to side-effects such as adding a new work to the flow.

finalize()

This method is called when the flow is completed. Here we write the final results in the “zprfrohl_results.json” file in the outdata directory of the flow

eph_flows Module

Flows for electron-phonon calculations (high-level interface)

class abipy.flowtk.eph_flows.EphPotFlow(workdir, manager=None, pickle_protocol=- 1, remove=False)

Bases: abipy.flowtk.flows.Flow

This flow computes the e-ph scattering potentials on a q-mesh defined by ngqpt and a list of q-points (usually a q-path) specified by the user. The DFPT potentials on the q-mesh are merged in the DVDB located in the outdata of the second work while the DFPT potentials on the q-path are merged in the DVDB located in the outdata of the third work. These DVDB files are then passed to the EPH code to compute the average over the unit cell of the periodic part of the scattering potentials as a function of q. Results are stored in the V1QAVG.nc files of the outdata of the tasks in the fourth work.

classmethod from_scf_input(workdir, scf_input, ngqpt, qbounds, ndivsm=5, with_becs=True, ddk_tolerance=None, prepgkk=0, manager=None)

Build the flow from an input file representing a GS calculation.

Parameters

• workdir – Working directory.

• scf_input – Input for the GS SCF run.

• ngqpt – 3 integers defining the q-mesh.

• qbounds – List of boundaries defining the q-path used for the computation of the GKQ files. The q-path is automatically generated using ndivsm and the reciprocal-space metric. If ndivsm is 0, the code assumes that qbounds contains the full list of q-points and no pre-processing is performed.

• ndivsm – Number of points in the smallest segment of the path defined by qbounds. Use 0 to pass list of q-points.

• with_becs – Activate calculation of Electric field and Born effective charges.

• ddk_tolerance – dict {“varname”: value} with the tolerance used in the DDK run if with_becs.

• prepgkk – 1 to activate computation of all 3 * natom perts (debugging option).

• manager – abipy.flowtk.tasks.TaskManager object.

class abipy.flowtk.eph_flows.GkqPathFlow(workdir, manager=None, pickle_protocol=- 1, remove=False)

Bases: abipy.flowtk.flows.Flow

This flow computes the gkq e-ph matrix elements <k+q|Delta V_q|k> for a list of q-points (usually a q-path). The results are stored in the GKQ.nc files for the different q-points. These files can be used to analyze the behaviour of the e-ph matrix elements as a function of qpts with the the objects provided by the abipy.eph.gkq module. It is also possible to compute the e-ph matrix elements using the interpolated DFPT potentials if test_ft_interpolation is set to True.

classmethod from_scf_input(workdir, scf_input, ngqpt, qbounds, ndivsm=5, with_becs=True, ddk_tolerance=None, test_ft_interpolation=False, prepgkk=0, manager=None)

Build the flow from an input file representing a GS calculation.

Parameters

• workdir – Working directory.

• scf_input – Input for the GS SCF run.

• ngqpt – 3 integers defining the q-mesh.

• qbounds – List of boundaries defining the q-path used for the computation of the GKQ files. The q-path is automatically generated using ndivsm and the reciprocal-space metric. If ndivsm is 0, the code assumes that qbounds contains the full list of q-points and no pre-processing is performed.

• ndivsm – Number of points in the smallest segment of the path defined by qbounds. Use 0 to pass list of q-points.

• with_becs – Activate calculation of Electric field and Born effective charges.

• ddk_tolerance – dict {“varname”: value} with the tolerance used in the DDK run if with_becs.

• test_ft_interpolation – True to add an extra Work in which the GKQ files are computed using the interpolated DFPT potentials and the q-mesh defined by ngqpt. The quality of the interpolation depends on the convergence of the BECS, epsinf and ngqpt.

• prepgkk – 1 to activate computation of all 3 * natom perts (debugging option).

• manager – abipy.flowtk.tasks.TaskManager object.

events Module

This module defines the events signaled by abinit during the execution. It also provides a parser to extract these events form the main output file and the log file.

class abipy.flowtk.events.EventsParser

Bases: object

Parses the output or the log file produced by ABINIT and extract the list of events.

Error

alias of abipy.flowtk.events.EventsParserError

parse(filename, verbose=0)

Parse the given file. Return EventReport.

report_exception(filename, exc)

This method is used when self.parser raises an Exception so that we can report a customized EventReport object with info the exception.

abipy.flowtk.events.get_event_handler_classes(categories=None)

Return the list of handler classes.

class abipy.flowtk.events.ScfConvergenceWarning(src_file, src_line, message)

Bases: abipy.flowtk.events.AbinitCriticalWarning

Warning raised when the GS SCF cycle did not converge.

yaml_tag: Any = '!ScfConvergenceWarning'

class abipy.flowtk.events.NscfConvergenceWarning(src_file, src_line, message)

Bases: abipy.flowtk.events.AbinitCriticalWarning

Warning raised when the GS NSCF cycle did not converge.

yaml_tag: Any = '!NscfConvergenceWarning'

class abipy.flowtk.events.RelaxConvergenceWarning(src_file, src_line, message)

Bases: abipy.flowtk.events.AbinitCriticalWarning

Warning raised when the structural relaxation did not converge.

yaml_tag: Any = '!RelaxConvergenceWarning'

class abipy.flowtk.events.Correction(handler, actions, event, reset=False)

Bases: monty.json.MSONable

as_dict()

A JSON serializable dict representation of an object.

classmethod from_dict(d)

Parameters

d – Dict representation.

Returns

MSONable class.

class abipy.flowtk.events.DilatmxError(src_file, src_line, message)

Bases: abipy.flowtk.events.AbinitError

This Error occurs in variable cell calculations when the increase in the unit cell volume is too large.

yaml_tag: Any = '!DilatmxError'

class abipy.flowtk.events.DilatmxErrorHandler(max_dilatmx=1.3)

Bases: abipy.flowtk.events.ErrorHandler

Handle DilatmxError. Abinit produces a netcdf file with the last structure before aborting The handler changes the structure in the input with the last configuration and modify the value of dilatmx.

event_class

alias of abipy.flowtk.events.DilatmxError

can_change_physics = False

as_dict()

Basic implementation of as_dict if __init__ has no arguments. Subclasses may need to overwrite.

classmethod from_dict(d)

Basic implementation of from_dict if __init__ has no arguments. Subclasses may need to overwrite.

handle_task_event(task, event)

Method to handle Abinit events.

Parameters

• task – Task object.

• event – AbinitEvent found in the log file.

Returns

0 if no action has been applied, 1 if the problem has been fixed.

handle_input_event(abiinput, outdir, event)

flows Module

A Flow is a container for Works, and works consist of tasks. Flows are the final objects that can be dumped directly to a pickle file on disk Flows are executed using abirun (abipy).

class abipy.flowtk.flows.Flow(workdir, manager=None, pickle_protocol=- 1, remove=False)

Bases: abipy.flowtk.nodes.Node, abipy.flowtk.works.NodeContainer, monty.json.MSONable

This object is a container of work. Its main task is managing the possible inter-dependencies among the work and the creation of dynamic workflows that are generated by callbacks registered by the user.

creation_date

String with the creation_date

pickle_protocol

Protocol for Pickle database (default: -1 i.e. latest protocol)

Important methods for constructing flows:

register_work: register (add) a work to the flow resister_task: register a work that contains only this task returns the work allocate: propagate the workdir and manager of the flow to all the registered tasks build: build_and_pickle_dump:

VERSION = '0.1'

PICKLE_FNAME = '__AbinitFlow__.pickle'

Error

alias of abipy.flowtk.flows.FlowError

Results

alias of abipy.flowtk.flows.FlowResults

classmethod from_inputs(workdir, inputs, manager=None, pickle_protocol=-1, task_class=<class 'abipy.flowtk.tasks.ScfTask'>, work_class=<class 'abipy.flowtk.works.Work'>, remove=False)

Construct a simple flow from a list of inputs. The flow contains a single Work with tasks whose class is given by task_class.

Warning

Don’t use this interface if you have dependencies among the tasks.

Parameters

• workdir – String specifying the directory where the works will be produced.

• inputs – List of inputs.

• manager – abipy.flowtk.tasks.TaskManager object responsible for the submission of the jobs. If manager is None, the object is initialized from the yaml file located either in the working directory or in the user configuration dir.

• pickle_protocol – Pickle protocol version used for saving the status of the object. -1 denotes the latest version supported by the python interpreter.

• task_class – The class of the abipy.flowtk.tasks.Task.

• work_class – The class of the abipy.flowtk.works.Work.

• remove – attempt to remove working directory workdir if directory already exists.

classmethod as_flow(obj)

Convert obj into a Flow. Accepts filepath, dict, or Flow object.

as_dict(**kwargs)

JSON serialization, note that we only need to save a string with the working directory since the object will be reconstructed from the pickle file located in workdir

to_dict(**kwargs)

JSON serialization, note that we only need to save a string with the working directory since the object will be reconstructed from the pickle file located in workdir

classmethod from_dict(d, **kwargs)

Reconstruct the flow from the pickle file.

classmethod temporary_flow(manager=None, workdir=None)

Return a Flow in a temporary directory. Useful for unit tests.

set_workdir(workdir, chroot=False)

Set the working directory. Cannot be set more than once unless chroot is True

reload()

Reload the flow from the pickle file. Used when we are monitoring the flow executed by the scheduler. In this case, indeed, the flow might have been changed by the scheduler and we have to reload the new flow in memory.

classmethod pickle_load(filepath, spectator_mode=True, remove_lock=False)

Loads the object from a pickle file and performs initial setup.

Parameters

• filepath – Filename or directory name. It filepath is a directory, we scan the directory tree starting from filepath and we read the first pickle database. Raise RuntimeError if multiple databases are found.

• spectator_mode – If True, the nodes of the flow are not connected by signals. This option is usually used when we want to read a flow in read-only mode and we want to avoid callbacks that can change the flow.

• remove_lock – True to remove the file lock if any (use it carefully).

classmethod from_file(filepath, spectator_mode=True, remove_lock=False)

Loads the object from a pickle file and performs initial setup.

Parameters

• filepath – Filename or directory name. It filepath is a directory, we scan the directory tree starting from filepath and we read the first pickle database. Raise RuntimeError if multiple databases are found.

• spectator_mode – If True, the nodes of the flow are not connected by signals. This option is usually used when we want to read a flow in read-only mode and we want to avoid callbacks that can change the flow.

• remove_lock – True to remove the file lock if any (use it carefully).

classmethod pickle_loads(s)

Reconstruct the flow from a string.

get_panel(**kwargs)

Build panel with widgets to interact with the abipy.flowtk.flows.Flow either in a notebook or in panel app.

set_pyfile(pyfile)

Set the path of the python script used to generate the flow.

property pyfile

Absolute path of the python script used to generate the flow. Set by set_pyfile

property pid_file

The path of the pid file created by PyFlowScheduler.

property has_scheduler

True if there’s a scheduler running the flow.

check_pid_file()

This function checks if we are already running the abipy.flowtk.flows.Flow with a PyFlowScheduler. Raises: Flow.Error if the pid file of the scheduler exists.

property pickle_file

The path of the pickle file.

property mongo_id

validate_json_schema()

Validate the JSON schema. Return list of errors.

get_mongo_info()

Return a JSON dictionary with information on the flow. Mainly used for constructing the info section in FlowEntry. The default implementation is empty. Subclasses must implement it

mongo_assimilate()

This function is called by client code when the flow is completed Return a JSON dictionary with the most important results produced by the flow. The default implementation is empty. Subclasses must implement it

property works

List of abipy.flowtk.works.Work objects contained in self..

property all_ok

True if all the tasks in works have reached S_OK.

property num_tasks

Total number of tasks

property errored_tasks

List of errored tasks.

property num_errored_tasks

The number of tasks whose status is S_ERROR.

property unconverged_tasks

List of unconverged tasks.

property num_unconverged_tasks

The number of tasks whose status is S_UNCONVERGED.

property status_counter

Returns a Counter object that counts the number of tasks with given status (use the string representation of the status as key).

property ncores_reserved

Returns the number of cores reserved in this moment. A core is reserved if the task is not running but we have submitted the task to the queue manager.

property ncores_allocated

Returns the number of cores allocated in this moment. A core is allocated if it’s running a task or if we have submitted a task to the queue manager but the job is still pending.

property ncores_used

Returns the number of cores used in this moment. A core is used if there’s a job that is running on it.

property has_chrooted

Returns a string that evaluates to True if we have changed the workdir for visualization purposes e.g. we are using sshfs. to mount the remote directory where the Flow is located. The string gives the previous workdir of the flow.

chroot(new_workdir)

Change the workir of the abipy.flowtk.flows.Flow. Mainly used for allowing the user to open the GUI on the local host and access the flow from remote via sshfs.

Note

Calling this method will make the flow go in read-only mode.

groupby_status()

Returns a ordered dictionary mapping the task status to the list of named tuples (task, work_index, task_index).

groupby_task_class()

Returns a dictionary mapping the task class to the list of tasks in the flow

iflat_nodes(status=None, op='==', nids=None)

Generators that produces a flat sequence of nodes. if status is not None, only the tasks with the specified status are selected. nids is an optional list of node identifiers used to filter the nodes.

node_from_nid(nid)

Return the node in the Flow with the given nid identifier

iflat_tasks_wti(status=None, op='==', nids=None)

Generator to iterate over all the tasks of the Flow. :Yields: (task, work_index, task_index)

If status is not None, only the tasks whose status satisfies the condition (task.status op status) are selected status can be either one of the flags defined in the abipy.flowtk.tasks.Task class (e.g Task.S_OK) or a string e.g “S_OK” nids is an optional list of node identifiers used to filter the tasks.

iflat_tasks(status=None, op='==', nids=None)

Generator to iterate over all the tasks of the abipy.flowtk.flows.Flow.

If status is not None, only the tasks whose status satisfies the condition (task.status op status) are selected status can be either one of the flags defined in the abipy.flowtk.tasks.Task class (e.g Task.S_OK) or a string e.g “S_OK” nids is an optional list of node identifiers used to filter the tasks.

abivalidate_inputs()

Run ABINIT in dry mode to validate all the inputs of the flow.

Returns

(isok, tuples)

isok is True if all inputs are ok. tuples is List of namedtuple objects, one for each task in the flow. Each namedtuple has the following attributes:

retcode: Return code. 0 if OK. log_file: log file of the Abinit run, use log_file.read() to access its content. stderr_file: stderr file of the Abinit run. use stderr_file.read() to access its content.

Raises

RuntimeError –

check_dependencies()

Test the dependencies of the nodes for possible deadlocks.

find_deadlocks()

This function detects deadlocks

Returns

deadlocks, runnables, running

Return type

named tuple with the tasks grouped in

check_status(**kwargs)

Check the status of the works in self.

Parameters

• show – True to show the status of the flow.

• kwargs – keyword arguments passed to show_status

property status

The status of the abipy.flowtk.flows.Flow i.e. the minimum of the status of its tasks and its works

fix_abicritical()

This function tries to fix critical events originating from ABINIT. Returns the number of tasks that have been fixed.

fix_queue_critical()

This function tries to fix critical events originating from the queue submission system.

Returns the number of tasks that have been fixed.

show_info(**kwargs)

Print info on the flow i.e. total number of tasks, works, tasks grouped by class.

Example

Task Class Number ———— ——– ScfTask 1 NscfTask 1 ScrTask 2 SigmaTask 6

compare_abivars(varnames, nids=None, wslice=None, printout=False, with_colors=False)

Print the input of the tasks to the given stream.

Parameters

• varnames – List of Abinit variables. If not None, only the variable in varnames are selected and printed.

• nids – List of node identifiers. By defaults all nodes are shown

• wslice – Slice object used to select works.

• printout – True to print dataframe.

• with_colors – True if task status should be colored.

get_dims_dataframe(nids=None, printout=False, with_colors=False)

Analyze output files produced by the tasks. Print pandas DataFrame with dimensions.

Parameters

• nids – List of node identifiers. By defaults all nodes are shown

• printout – True to print dataframe.

• with_colors – True if task status should be colored.

compare_structures(nids=None, with_spglib=False, what='io', verbose=0, precision=3, printout=False, with_colors=False)

Analyze structures of the tasks (input and output structures if it’s a relaxation task. Print pandas DataFrame

Parameters

• nids – List of node identifiers. By defaults all nodes are shown

• with_spglib – If True, spglib is invoked to get the spacegroup symbol and number

• what (str) – “i” for input structures, “o” for output structures.

• precision – Floating point output precision (number of significant digits). This is only a suggestion

• printout – True to print dataframe.

• with_colors – True if task status should be colored.

compare_ebands(nids=None, with_path=True, with_ibz=True, with_spglib=False, verbose=0, precision=3, printout=False, with_colors=False)

Analyze electron bands produced by the tasks. Return pandas DataFrame and abipy.electrons.ebands.ElectronBandsPlotter.

Parameters

• nids – List of node identifiers. By default, all nodes are shown

• with_path – Select files with ebands along k-path.

• with_ibz – Select files with ebands in the IBZ.

• with_spglib – If True, spglib is invoked to get the spacegroup symbol and number

• precision – Floating point output precision (number of significant digits). This is only a suggestion

• printout – True to print dataframe.

• with_colors – True if task status should be colored.

Return: (df, ebands_plotter)

compare_hist(nids=None, with_spglib=False, verbose=0, precision=3, printout=False, with_colors=False)

Analyze HIST nc files produced by the tasks. Print pandas DataFrame with final results. Return: (df, hist_plotter)

Parameters

• nids – List of node identifiers. By defaults all nodes are shown

• with_spglib – If True, spglib is invoked to get the spacegroup symbol and number

• precision – Floating point output precision (number of significant digits). This is only a suggestion

• printout – True to print dataframe.

• with_colors – True if task status should be colored.

show_summary(**kwargs)

Print a short summary with the status of the flow and a counter task_status –> number_of_tasks

Parameters

stream – File-like object, Default: sys.stdout

Example

Status Count ——— ——- Completed 10

<Flow, node_id=27163, workdir=flow_gwconv_ecuteps>, num_tasks=10, all_ok=True

show_status(**kwargs)

Report the status of the works and the status of the different tasks on the specified stream.

Parameters

• stream – File-like object, Default: sys.stdout

• nids – List of node identifiers. By defaults all nodes are shown

• wslice – Slice object used to select works.

• verbose – Verbosity level (default 0). > 0 to show only the works that are not finalized.

show_events(status=None, nids=None, stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Print the Abinit events (ERRORS, WARNIING, COMMENTS) to stdout

Parameters

• status – if not None, only the tasks with this status are select

• nids – optional list of node identifiers used to filter the tasks.

• stream – File-like object, Default: sys.stdout

show_corrections(status=None, nids=None, stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Show the corrections applied to the flow at run-time.

Parameters

• status – if not None, only the tasks with this status are select.

• nids – optional list of node identifiers used to filter the tasks.

• stream – File-like object, Default: sys.stdout

Return: The number of corrections found.

show_history(status=None, nids=None, full_history=False, metadata=False, stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Print the history of the flow to stream

Parameters

• status – if not None, only the tasks with this status are select

• full_history – Print full info set, including nodes with an empty history.

• nids – optional list of node identifiers used to filter the tasks.

• metadata – print history metadata (experimental)

• stream – File-like object, Default: sys.stdout

show_inputs(varnames=None, nids=None, wslice=None, stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Print the input of the tasks to the given stream.

Parameters

• varnames – List of Abinit variables. If not None, only the variable in varnames are selected and printed.

• nids – List of node identifiers. By defaults all nodes are shown

• wslice – Slice object used to select works.

• stream – File-like object, Default: sys.stdout

listext(ext, stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Print to the given stream a table with the list of the output files with the given ext produced by the flow.

select_tasks(nids=None, wslice=None, task_class=None)

Return a list with a subset of tasks.

Parameters

• nids – List of node identifiers.

• wslice – Slice object used to select works.

• task_class – String or class used to select tasks. Ignored if None.

Note

nids and wslice are mutually exclusive. If no argument is provided, the full list of tasks is returned.

get_task_scfcycles(nids=None, wslice=None, task_class=None, exclude_ok_tasks=False)

Return list of (taks, scfcycle) tuples for all the tasks in the flow with a SCF algorithm e.g. electronic GS-SCF iteration, DFPT-SCF iterations etc.

Parameters

• nids – List of node identifiers.

• wslice – Slice object used to select works.

• task_class – String or class used to select tasks. Ignored if None.

• exclude_ok_tasks – True if only running tasks should be considered.

Returns

List of ScfCycle subclass instances.

show_tricky_tasks(verbose=0, stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Print list of tricky tasks i.e. tasks that have been restarted or launched more than once or tasks with corrections.

Parameters

• verbose – Verbosity level. If > 0, task history and corrections (if any) are printed.

• stream – File-like object. Default: sys.stdout

inspect(nids=None, wslice=None, **kwargs)

Inspect the tasks (SCF iterations, Structural relaxation …) and produces matplotlib plots.

Parameters

• nids – List of node identifiers.

• wslice – Slice object used to select works.

• kwargs – keyword arguments passed to task.inspect method.

Note

nids and wslice are mutually exclusive. If nids and wslice are both None, all tasks in self are inspected.

Returns

List of matplotlib figures.

get_results(**kwargs)

get_dict_for_mongodb_queries()

This function returns a dictionary with the attributes that will be put in the mongodb document to facilitate the query. Subclasses may want to replace or extend the default behaviour.

look_before_you_leap()

This method should be called before running the calculation to make sure that the most important requirements are satisfied.

Returns

List of strings with inconsistencies/errors.

property has_db

True if flow uses MongoDB to store the results.

db_insert()

Insert results in the MongDB database.

tasks_from_nids(nids)

Return the list of tasks associated to the given list of node identifiers (nids).

Note

Invalid ids are ignored

wti_from_nids(nids)

Return the list of (w, t) indices from the list of node identifiers nids.

open_files(what='o', status=None, op='==', nids=None, editor=None)

Open the files of the flow inside an editor (command line interface).

Parameters

• what –

  string with the list of characters selecting the file type Possible choices:

  i ==> input_file, o ==> output_file, f ==> files_file, j ==> job_file, l ==> log_file, e ==> stderr_file, q ==> qout_file, all ==> all files.

• status – if not None, only the tasks with this status are select

• op – status operator. Requires status. A task is selected if task.status op status evaluates to true.

• nids – optional list of node identifiers used to filter the tasks.

• editor – Select the editor. None to use the default editor ($EDITOR shell env var)

parse_timing(nids=None)

Parse the timer data in the main output file(s) of Abinit. Requires timopt /= 0 in the input file, usually timopt = -1.

Parameters

nids – optional list of node identifiers used to filter the tasks.

Return: AbinitTimerParser instance, None if error.

show_abierrors(nids=None, stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Write to the given stream the list of ABINIT errors for all tasks whose status is S_ABICRITICAL.

Parameters

• nids – optional list of node identifiers used to filter the tasks.

• stream – File-like object. Default: sys.stdout

show_qouts(nids=None, stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Write to the given stream the content of the queue output file for all tasks whose status is S_QCRITICAL.

Parameters

• nids – optional list of node identifiers used to filter the tasks.

• stream – File-like object. Default: sys.stdout

debug(status=None, nids=None, stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

This method is usually used when the flow didn’t completed succesfully It analyzes the files produced the tasks to facilitate debugging. Info are printed to stdout.

Parameters

• status – If not None, only the tasks with this status are selected

• nids – optional list of node identifiers used to filter the tasks.

• stream – File-like object. Default: sys.stdout

cancel(nids=None)

Cancel all the tasks that are in the queue. nids is an optional list of node identifiers used to filter the tasks.

Returns

Number of jobs cancelled, negative value if error

get_njobs_in_queue(username=None)

returns the number of jobs in the queue, None when the number of jobs cannot be determined.

Parameters

username – (str) the username of the jobs to count (default is to autodetect)

rmtree(ignore_errors=False, onerror=None)

Remove workdir (same API as shutil.rmtree).

rm_and_build()

Remove the workdir and rebuild the flow.

build(*args, **kwargs)

Make directories and files of the Flow.

build_and_pickle_dump(abivalidate=False)

Build dirs and file of the Flow and save the object in pickle format. Returns 0 if success

Parameters

abivalidate – If True, all the input files are validate by calling the abinit parser. If the validation fails, ValueError is raise.

pickle_dump()

Save the status of the object in pickle format. Returns 0 if success

pickle_dumps(protocol=None)

Return a string with the pickle representation. protocol selects the pickle protocol. self.pickle_protocol is used if protocol is None

register_task(input, deps=None, manager=None, task_class=None, append=False)

Utility function that generates a Work made of a single task

Parameters

• input – abipy.abio.inputs.AbinitInput

• deps – List of Dependency objects specifying the dependency of this node. An empy list of deps implies that this node has no dependencies.

• manager – The abipy.flowtk.tasks.TaskManager responsible for the submission of the task. If manager is None, we use the abipy.flowtk.tasks.TaskManager specified during the creation of the work.

• task_class – Task subclass to instantiate. Default: abipy.flowtk.tasks.AbinitTask

• append – If true, the task is added to the last work (a new Work is created if flow is empty)

Returns

The generated abipy.flowtk.works.Work for the task, work[0] is the actual task.

new_work(deps=None, manager=None, workdir=None)

Helper function to add a new empty abipy.flowtk.works.Work and add it to the internal list. Client code is responsible for filling the new work.

Parameters

• deps – List of Dependency objects specifying the dependency of this node. An empy list of deps implies that this node has no dependencies.

• manager – The abipy.flowtk.tasks.TaskManager responsible for the submission of the task. If manager is None, we use the TaskManager specified during the creation of the work.

• workdir – The name of the directory used for the abipy.flowtk.works.Work.

Returns

The registered abipy.flowtk.works.Work.

register_work(work, deps=None, manager=None, workdir=None)

Register a new abipy.flowtk.works.Work and add it to the internal list, taking into account possible dependencies.

Parameters

• work – abipy.flowtk.works.Work object.

• deps – List of Dependency objects specifying the dependency of this node. An empy list of deps implies that this node has no dependencies.

• manager – The abipy.flowtk.tasks.TaskManager responsible for the submission of the task. If manager is None, we use the TaskManager specified during the creation of the work.

• workdir – The name of the directory used for the abipy.flowtk.works.Work.

Returns

The registered abipy.flowtk.works.Work.

register_work_from_cbk(cbk_name, cbk_data, deps, work_class, manager=None)

Registers a callback function that will generate the Task of the Work.

Parameters

• cbk_name – Name of the callback function (must be a bound method of self)

• cbk_data – Additional data passed to the callback function.

• deps – List of Dependency objects specifying the dependency of the work.

• work_class – abipy.flowtk.works.Work class to instantiate.

• manager – The abipy.flowtk.tasks.TaskManager responsible for the submission of the task. If manager is None, we use the TaskManager specified during the creation of the abipy.flowtk.flows.Flow.

Returns

The abipy.flowtk.works.Work that will be finalized by the callback.

property allocated

Numer of allocations. Set by allocate.

allocate(workdir=None, use_smartio=False)

Allocate the Flow i.e. assign the workdir and (optionally) the abipy.flowtk.tasks.TaskManager to the different tasks in the Flow.

Parameters

workdir – Working directory of the flow. Must be specified here if we haven’t initialized the workdir in the __init__.

Returns

self

use_smartio()

This function should be called when the entire Flow has been built. It tries to reduce the pressure on the hard disk by using Abinit smart-io capabilities for those files that are not needed by other nodes. Smart-io means that big files (e.g. WFK) are written only if the calculation is unconverged so that we can restart from it. No output is produced if convergence is achieved.

Return: self

show_dependencies(stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Writes to the given stream the ASCII representation of the dependency tree.

on_all_ok()

This method is called when all the works in the flow have reached S_OK. This method shall return True if the calculation is completed or False if the execution should continue due to side-effects such as adding a new work to the flow.

This methods allows subclasses to implement customized logic such as extending the flow by adding new works. The flow has an internal counter: on_all_ok_num_calls that shall be incremented by client code when subclassing this method. This counter can be used to decide if futher actions are needed or not.

An example of flow that adds a new work (only once) when all_ok is reached for the first time:

def on_all_ok(self):

if self.on_all_ok_num_calls > 0: return True self.on_all_ok_num_calls += 1

implement_logic_to_create_new_work

self.register_work(work) self.allocate() self.build_and_pickle_dump()

return False # The scheduler will keep on running the flow.

on_dep_ok(signal, sender)

finalize()

This method is called when the flow is completed. Return 0 if success

set_garbage_collector(exts=None, policy='task')

Enable the garbage collector that will remove the big output files that are not needed.

Parameters

• exts – string or list with the Abinit file extensions to be removed. A default is provided if exts is None

• policy – Either flow or task. If policy is set to ‘task’, we remove the output files as soon as the task reaches S_OK. If ‘flow’, the files are removed only when the flow is finalized. This option should be used when we are dealing with a dynamic flow with callbacks generating other tasks since a abipy.flowtk.tasks.Task might not be aware of its children when it reached S_OK.

connect_signals()

Connect the signals within the Flow. The Flow is responsible for catching the important signals raised from its works.

disconnect_signals()

Disable the signals within the Flow.

show_receivers(sender=None, signal=None)

set_spectator_mode(mode=True)

When the flow is in spectator_mode, we have to disable signals, pickle dump and possible callbacks A spectator can still operate on the flow but the new status of the flow won’t be saved in the pickle file. Usually the flow is in spectator mode when we are already running it via the scheduler or other means and we should not interfere with its evolution. This is the reason why signals and callbacks must be disabled. Unfortunately preventing client-code from calling methods with side-effects when the flow is in spectator mode is not easy (e.g. flow.cancel will cancel the tasks submitted to the queue and the flow used by the scheduler won’t see this change!

rapidfire(check_status=True, max_nlaunch=- 1, max_loops=1, sleep_time=5, **kwargs)

Use PyLauncher to submits tasks in rapidfire mode. kwargs contains the options passed to the launcher.

Parameters

• check_status –

• max_nlaunch – Maximum number of launches. default: no limit.

• max_loops – Maximum number of loops

• sleep_time – seconds to sleep between rapidfire loop iterations

Return: Number of tasks submitted.

single_shot(check_status=True, **kwargs)

Use PyLauncher to submit one task. kwargs contains the options passed to the launcher.

Return: Number of tasks submitted.

make_scheduler(**kwargs)

Build and return a PyFlowScheduler to run the flow.

Parameters

kwargs – if empty we use the user configuration file. if filepath in kwargs we init the scheduler from filepath. else pass kwargs to PyFlowScheduler __init__ method.

batch(timelimit=None)

Run the flow in batch mode, return exit status of the job script. Requires a manager.yml file and a batch_adapter adapter.

Parameters

• timelimit – Time limit (int with seconds or string with time given with the slurm convention:

• "days-hours – minutes:seconds”). If timelimit is None, the default value specified in the

• entry of manager.yml is used. (batch_adapter) –

make_light_tarfile(name=None)

Lightweight tarball file. Mainly used for debugging. Return the name of the tarball file.

make_tarfile(name=None, max_filesize=None, exclude_exts=None, exclude_dirs=None, verbose=0, **kwargs)

Create a tarball file.

Parameters

• name – Name of the tarball file. Set to os.path.basename(flow.workdir) + “tar.gz”` if name is None.

• max_filesize (int or string with unit) – a file is included in the tar file if its size <= max_filesize Can be specified in bytes e.g. max_files=1024 or with a string with unit e.g. max_filesize=”1 Mb”. No check is done if max_filesize is None.

• exclude_exts – List of file extensions to be excluded from the tar file.

• exclude_dirs – List of directory basenames to be excluded.

• verbose (int) – Verbosity level.

• kwargs – keyword arguments passed to the TarFile constructor.

Returns: The name of the tarfile.

get_graphviz(engine='automatic', graph_attr=None, node_attr=None, edge_attr=None)

Generate flow graph in the DOT language.

Parameters

• engine – Layout command used. [‘dot’, ‘neato’, ‘twopi’, ‘circo’, ‘fdp’, ‘sfdp’, ‘patchwork’, ‘osage’]

• graph_attr – Mapping of (attribute, value) pairs for the graph.

• node_attr – Mapping of (attribute, value) pairs set for all nodes.

• edge_attr – Mapping of (attribute, value) pairs set for all edges.

Returns: graphviz.Digraph <https://graphviz.readthedocs.io/en/stable/api.html#digraph>

graphviz_imshow(ax=None, figsize=None, dpi=300, fmt='png', **kwargs)

Generate flow graph in the DOT language and plot it with matplotlib.

Parameters

• ax – matplotlib.axes.Axes or None if a new figure should be created.

• figsize – matplotlib figure size (None to use default)

• dpi – DPI value.

• fmt – Select format for output image

Return: matplotlib.figure.Figure

Keyword arguments controlling the display of the figure:

kwargs	Meaning
title	Title of the plot (Default: None).
show	True to show the figure (default: True).
savefig	“abc.png” or “abc.eps” to save the figure to a file.
size_kwargs	Dictionary with options passed to fig.set_size_inches e.g. size_kwargs=dict(w=3, h=4)
tight_layout	True to call fig.tight_layout (default: False)
ax_grid	True (False) to add (remove) grid from all axes in fig. Default: None i.e. fig is left unchanged.
ax_annotate	Add labels to subplots e.g. (a), (b). Default: False
fig_close	Close figure. Default: False.

plot_networkx(mode='network', with_edge_labels=False, ax=None, arrows=False, node_size='num_cores', node_label='name_class', layout_type='spring', **kwargs)

Use networkx to draw the flow with the connections among the nodes and the status of the tasks.

Parameters

• mode – networkx to show connections, status to group tasks by status.

• with_edge_labels – True to draw edge labels.

• ax – matplotlib.axes.Axes or None if a new figure should be created.

• arrows – if True draw arrowheads.

• node_size – By default, the size of the node is proportional to the number of cores used.

• node_label – By default, the task class is used to label node.

• layout_type – Get positions for all nodes using layout_type. e.g. pos = nx.spring_layout(g)

Warning

Requires networkx package.

Keyword arguments controlling the display of the figure:

kwargs	Meaning
title	Title of the plot (Default: None).
show	True to show the figure (default: True).
savefig	“abc.png” or “abc.eps” to save the figure to a file.
size_kwargs	Dictionary with options passed to fig.set_size_inches e.g. size_kwargs=dict(w=3, h=4)
tight_layout	True to call fig.tight_layout (default: False)
ax_grid	True (False) to add (remove) grid from all axes in fig. Default: None i.e. fig is left unchanged.
ax_annotate	Add labels to subplots e.g. (a), (b). Default: False
fig_close	Close figure. Default: False.

write_open_notebook(foreground)

Generate an ipython notebook and open it in the browser. Return system exit code.

class abipy.flowtk.flows.G0W0WithQptdmFlow(workdir, scf_input, nscf_input, scr_input, sigma_inputs, manager=None)

Bases: abipy.flowtk.flows.Flow

cbk_qptdm_workflow(cbk)

This callback is executed by the flow when bands_work.nscf_task reaches S_OK.

It computes the list of q-points for the W(q,G,G’), creates nqpt tasks in the second work (QptdmWork), and connect the signals.

abipy.flowtk.flows.bandstructure_flow(workdir, scf_input, nscf_input, dos_inputs=None, manager=None, flow_class=<class 'abipy.flowtk.flows.Flow'>, allocate=True)

Build a abipy.flowtk.flows.Flow for band structure calculations.

Parameters

• workdir – Working directory.

• scf_input – Input for the GS SCF run.

• nscf_input – Input for the NSCF run (band structure run).

• dos_inputs – Input(s) for the NSCF run (dos run).

• manager – abipy.flowtk.tasks.TaskManager object used to submit the jobs. Initialized from manager.yml if manager is None.

• flow_class – Flow subclass

• allocate – True if the flow should be allocated before returning.

Returns: abipy.flowtk.flows.Flow object

abipy.flowtk.flows.g0w0_flow(workdir, scf_input, nscf_input, scr_input, sigma_inputs, manager=None, flow_class=<class 'abipy.flowtk.flows.Flow'>, allocate=True)

Build a abipy.flowtk.flows.Flow for one-shot $G_0W_0$ calculations.

Parameters

• workdir – Working directory.

• scf_input – Input for the GS SCF run.

• nscf_input – Input for the NSCF run (band structure run).

• scr_input – Input for the SCR run.

• sigma_inputs – List of inputs for the SIGMA run.

• flow_class – Flow class

• manager – abipy.flowtk.tasks.TaskManager object used to submit the jobs. Initialized from manager.yml if manager is None.

• allocate – True if the flow should be allocated before returning.

Returns: abipy.flowtk.flows.Flow object

gw Module

Works and Flows for GW calculations with Abinit.

class abipy.flowtk.gw.ScreeningWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

This work parallelizes the calculation of the q-points of the screening. It also provides the callback on_all_ok that calls the mrgscr tool to merge all the partial SCR files produced.

The final SCR file is made available in the outdata directory of the work. To use this SCR file as dependency of other tasks, use the syntax:

deps={scr_work: “SCR”}

Inheritance Diagram

classmethod from_nscf_task(nscf_task, scr_input, manager=None)

Construct a ScreeningWork from a abipy.flowtk.tasks.NscfTask object.

Parameters

• nscf_task – abipy.flowtk.tasks.NscfTask object.

• scr_input – abipy.abio.inputs.AbinitInput object representing a SCREENING calculation.

• manager – abipy.flowtk.tasks.TaskManager object.

classmethod from_wfk_filepath(wfk_filepath, scr_input, manager=None)

Construct a ScreeningWork from a WFK filepath and a screening input.

WARNING: the parameters reported in scr_input e.g. k-mesh, nband, ecut etc must be consisten with those used to generate the WFK file. No consistencty check is done at this level. You have been warned!

Parameters

• wfk_filepath – Path to the WFK file.

• scr_input – abipy.abio.inputs.AbinitInput object representing a SCREENING calculation.

• manager – abipy.flowtk.tasks.TaskManager object.

merge_scr_files(remove_scrfiles=True, verbose=0)

This method is called when all the q-points have been computed. It runs mrgscr in sequential on the local machine to produce the final SCR file in the outdir of the Work. If remove_scrfiles is True, the partial SCR files are removed after the merge.

on_all_ok()

This method is called when all the q-points have been computed. It runs mrgscr in sequential on the local machine to produce the final SCR file in the outdir of the Work.

gruneisen Module

Work for computing the Grüneisen parameters with finite differences of DFPT phonons.

WARNING: This code must be tested more carefully.

class abipy.flowtk.gruneisen.GruneisenWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

This work computes the Grüneisen parameters (derivative of frequencies wrt volume) using finite differences and the phonons obtained with the DFPT part of Abinit. The Anaddb input file needed to compute Grüneisen parameters will be generated in the outdata directory of the flow.

It is necessary to run three DFPT phonon calculations. One is calculated at the equilibrium volume and the remaining two are calculated at the slightly larger volume and smaller volume than the equilibrium volume. The unitcells at these volumes have to be fully relaxed under the constraint of each volume. This Work automates the entire procedure starting from an input for GS calculations.

classmethod from_gs_input(gs_inp, voldelta, ngqpt, tolerance=None, with_becs=False, ddk_tolerance=None, workdir=None, manager=None)

Build the work from an abipy.abio.inputs.AbinitInput representing a GS calculations.

Parameters

• gs_inp – abipy.abio.inputs.AbinitInput representing a GS calculation in the initial unit cell.

• voldelta – Absolute increment for unit cell volume. The three volumes are: [v0 - voldelta, v0, v0 + voldelta] where v0 is taken from gs_inp.structure.

• ngqpt – three integers defining the q-mesh for phonon calculations.

• tolerance – dict {“varname”: value} with the tolerance to be used in the phonon run. None to use AbiPy default.

• with_becs – Activate calculation of Electric field and Born effective charges.

• ddk_tolerance – dict {“varname”: value} with the tolerance used in the DDK run if with_becs. None to use AbiPy default.

on_all_ok()

This method is called once the Work is completed i.e. when all the tasks have reached status S_OK.

add_phonon_works_and_build()

Get relaxed structures from the tasks, build Phonons works with new structures. Add works to the flow and build new directories.

gs_works Module

Work subclasses related to GS calculations.

class abipy.flowtk.gs_works.EosWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

Work for the computation of the Equation of State The EOS is obtained by computing E(V) for several volumes around the input V0, The initial volumes are obtained by rescaling the input lattice vectors so that length proportions and angles are preserved. This guess is exact for cubic materials while other Bravais lattices require a constant-volume optimization of the cell geometry.

If lattice_type==”cubic” and atomic positions are fixed by symmetry. use can use move_atoms=False to perform standard GS-SCF calculations. In all the other cases, E(V) is obtained by relaxing the atomic positions at fixed volume.

The E(V) points are fitted at the end of the work and the results are saved in the eos_data.json file produced in the outdata directory. The file contains the energies, the volumes and the values of V0, B0, B1 obtained with different EOS models.

classmethod from_scf_input(scf_input, npoints=4, deltap_vol=0.25, ecutsm=0.5, move_atoms=True, manager=None)

Build a EosWork from an AbinitInput representing a SCF-GS calculation.

Parameters

• scf_input – AbinitInput for SCF-GS used as template to generate the other inputs.

• npoints – Number of volumes generated on the right (left) of the equilibrium volume The total number of points is therefore 2 * n + 1.

• deltap_vol – Step of the linear mesh given in relative percentage of the equilibrium volume The step is thus: v0 * deltap_vol / 100.

• ecutsm – Value of ecutsm input variable. If scf_input does not provide ecutsm, this value will be used else the vale in scf_input.

• move_atoms – If True, a structural relaxation of ions is performed for each volume This is needed if the atomic positions are non fixed by symmetry.

• manager – TaskManager instance. Use default if None.

Return: EosWork instance.

getnwrite_eosdata(write_json=True)

This method is called when all tasks reach S_OK. It reads the energies and the volumes from the GSR file, computes the EOS and produce a JSON file eos_data.json in outdata.

on_all_ok()

This method is called when all tasks reach S_OK. It reads the energies and the volumes from the GSR file, computes the EOS and produce a JSON file eos_data.json in outdata.

launcher Module

Tools for the submission of Tasks.

class abipy.flowtk.launcher.ScriptEditor

Bases: object

Simple editor that simplifies the writing of shell scripts

property shell

reset()

Reset the editor.

shebang()

Adds the shebang line.

declare_var(key, val)

Declare a env variable. If val is None the variable is unset.

declare_vars(d)

Declare the variables defined in the dictionary d.

export_envar(key, val)

Export an environment variable.

export_envars(env)

Export the environment variables contained in the dict env.

add_emptyline()

Add an empty line.

add_comment(comment)

Add a comment

load_modules(modules)

Load the list of specified modules.

load_module(module)

add_line(line)

add_lines(lines)

get_script_str(reset=True)

Returns a string with the script and reset the editor if reset is True

class abipy.flowtk.launcher.PyLauncher(flow, **kwargs)

Bases: object

This object handle the submission of the tasks contained in a abipy.flowtk.flows.Flow.

Error

alias of abipy.flowtk.launcher.PyLauncherError

single_shot()

Run the first Task than is ready for execution.

Returns

Number of jobs launched.

rapidfire(max_nlaunch=- 1, max_loops=1, sleep_time=5)

Keeps submitting Tasks until we are out of jobs or no job is ready to run.

Parameters

• max_nlaunch – Maximum number of launches. default: no limit.

• max_loops – Maximum number of loops

• sleep_time – seconds to sleep between rapidfire loop iterations

Returns

The number of tasks launched.

fetch_tasks_to_run()

Return the list of tasks that can be submitted. Empty list if no task has been found.

class abipy.flowtk.launcher.PyFlowScheduler(**kwargs)

Bases: object

This object schedules the submission of the tasks in a abipy.flowtk.flows.Flow. There are two types of errors that might occur during the execution of the jobs:

1. Python exceptions

2. Errors in the ab-initio code

Python exceptions are easy to detect and are usually due to a bug in the python code or random errors such as IOError. The set of errors in the ab-initio is much much broader. It includes wrong input data, segmentation faults, problems with the resource manager, etc. The flow tries to handle the most common cases but there’s still a lot of room for improvement. Note, in particular, that PyFlowScheduler will shutdown automatically in the following cases:

1. The number of python exceptions is > max_num_pyexcs

2. The number of task errors (i.e. the number of tasks whose status is S_ERROR) is > max_num_abierrs

3. The number of jobs launched becomes greater than (safety_ratio * total_number_of_tasks).

4. The scheduler will send an email to the user (specified by mailto) every remindme_s seconds. If the mail cannot be sent, the scheduler will shutdown automatically. This check prevents the scheduler from being trapped in an infinite loop.

YAML_FILE = 'scheduler.yml'

USER_CONFIG_DIR = '/Users/gmatteo/.abinit/abipy'

Error

alias of abipy.flowtk.launcher.PyFlowSchedulerError

classmethod autodoc()

classmethod from_file(filepath)

Read the configuration parameters from a Yaml file.

classmethod from_string(s)

Create an istance from string s containing a YAML dictionary.

classmethod from_user_config()

Initialize the PyFlowScheduler from the YAML file ‘scheduler.yml’. Search first in the working directory and then in the configuration directory of abipy.

Raises

RuntimeError –

property pid

The pid of the process associated to the scheduler.

property pid_file

Absolute path of the file with the pid. The file is located in the workdir of the flow

property flow

Flow.

property num_excs

Number of exceptions raised so far.

get_delta_etime()

Returns a timedelta object representing with the elapsed time.

add_flow(flow)

Add a flow to the scheduler.

start()

Starts the scheduler in a new thread. Returns 0 if success. In standalone mode, this method will block until there are no more scheduled jobs.

callback()

The function that will be executed by the scheduler.

cleanup()

Cleanup routine: remove the pid file and save the pickle database

shutdown(msg)

Shutdown the scheduler.

send_email(msg, tag=None)

Send an e-mail before completing the shutdown. Returns 0 if success.

lumi_works Module

Work subclasses for the computation of luminiscent properties.

class abipy.flowtk.lumi_works.LumiWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

This Work implements Fig 1 of https://arxiv.org/abs/2010.00423.

Client code is responsible for the preparation of the supercell and of the GS SCF input files with the fixed electronic occupations associated to the two configurations. By default, the work computes four total energies corresponding to the Ag, Ag*, Ae*, Ae configurations. Optionally, one can activate the computation of four electronic band structures and phonons for Ag and Ae. See docstring of from_scf_inputs for further info.

classmethod from_scf_inputs(gs_scf_inp, exc_scf_inp, relax_kwargs, ndivsm=0, nb_extra=10, tolwfr=1e-20, ngqpt=None, manager=None)

Parameters

• gs_scf_inp – abipy.abio.inputs.AbinitInput representing a GS SCF run for the ground-state.

• exc_scf_inp – abipy.abio.inputs.AbinitInput representing a GS SCF run for the excited-state.

• relax_kwargs – Dictonary with input variables to be added to gs_scf_inp and exc_scf_inp. when generating input files for structural relaxations.

• ndivsm – Activates the computation of band structure if different from zero. if > 0, it’s the number of divisions for the smallest segment of the path (Abinit variable). if < 0, it’s interpreted as the pymatgen line_density parameter in which the number of points in the segment is proportional to its length. Typical value: -20. This option is the recommended one if the k-path contains two high symmetry k-points that are very close as ndivsm > 0 may produce a very large number of wavevectors.

• nb_extra – Number of extra bands added to the input nband when computing band structures (ndivsm != 0).

• tolwfr – Tolerance on the residuals used for the NSCF band structure calculations.

• ngqpt – If not None, activates computation of phonons with DFPT using this q-mesh. Usually Gamma-only that is ngqpt = [1, 1, 1].

• manager – abipy.flowtk.tasks.TaskManager of the work. If None, the manager is initialized from the config file.

on_all_ok()

This method is called when all the tasks in the work have reached S_OK.

This is the section in which we implement most of the workflow logic at runtime. since we need to generate input files with relaxed structures.

mocks Module

Mock objects for unit tests.

abipy.flowtk.mocks.change_task_start(task, mocked_status='Error')

Return a AbinitTaskMockedStart object.

class abipy.flowtk.mocks.AbinitTaskMockedStart(input, workdir=None, manager=None, deps=None)

Bases: abipy.flowtk.tasks.AbinitTask

A Task whose status is always self.mocked_status.

start(**kwargs)

Starts the calculation by performing the following steps:

• build dirs and files

• call the _setup method

• execute the job file by executing/submitting the job script.

Main entry point for the Launcher.

kwargs	Meaning
autoparal	False to skip the autoparal step (default True)
exec_args	List of arguments passed to executable.

Returns

1 if task was started, 0 otherwise.

abipy.flowtk.mocks.infinite_flow(flow)

Return an InfiniteFlow.

class abipy.flowtk.mocks.InfiniteFlow(workdir, manager=None, pickle_protocol=- 1, remove=False)

Bases: abipy.flowtk.flows.Flow

A Flow that will never reach all_ok

check_status(**kwargs)

Check the status of the works in self.

Parameters

• show – True to show the status of the flow.

• kwargs – keyword arguments passed to show_status

netcdf Module

nodes Module

This module defines the Node base class inherited by Task, Work and Flow objects.

class abipy.flowtk.nodes.Status

Bases: int

This object is an integer representing the status of the Node.

classmethod as_status(obj)

Convert obj into Status.

classmethod from_string(s)

Return a Status instance from its string representation.

classmethod all_status_strings()

List of strings with all possible values status.

property is_critical

True if status is critical.

property color_opts

property colored

Return colorized text used to print the status if the stream supports it.

class abipy.flowtk.nodes.Dependency(node, exts=None)

Bases: object

This object describes the dependencies among the nodes of a calculation.

A Dependency consists of a Node that produces a list of products (files) that are used by the other nodes (Task or Work) to start the calculation. One usually creates the object by calling work.register

Example

# Register the SCF task in work. scf_task = work.register(scf_strategy)

# Register the NSCF calculation and its dependency on the SCF run via deps. nscf_task = work.register(nscf_strategy, deps={scf_task: “DEN”})

property info

property node

The abipy.flowtk.nodes.Node associated to the dependency.

property status

The status of the dependency, i.e. the status of the abipy.flowtk.nodes.Node.

products()

List of output files produces by self.

apply_getters(task)

This function is called when we specify the task dependencies with the syntax:

deps={node: “@property”}

In this case the task has to the get property from node before starting the calculation.

At present, the following properties are supported:

• @structure

connecting_vars()

Returns a dictionary with the variables that must be added to the input file in order to connect this abipy.flowtk.nodes.Node to its dependencies.

get_filepaths_and_exts()

Returns the paths of the output files produced by self and its extensions

class abipy.flowtk.nodes.Product(ext, path)

Bases: object

A product represents an output file produced by ABINIT instance. This file is needed to start another Task or another Work.

classmethod from_file(filepath)

Build a Product instance from a filepath.

property filepath

Absolute path of the file.

connecting_vars()

Returns a dictionary with the ABINIT variables that must be used to make the code use this file.

class abipy.flowtk.nodes.GridFsFile(path, fs_id=None, mode='b')

Bases: monty.collections.AttrDict

Information on a file that will stored in the MongoDb gridfs collection.

class abipy.flowtk.nodes.NodeResults(node, **kwargs)

Bases: dict, monty.json.MSONable

Dictionary used to store the most important results produced by a abipy.flowtk.nodes.Node.

JSON_SCHEMA = {'properties': {'exceptions': {'required': True, 'type': 'array'}, 'files': {'required': True, 'type': 'object'}, 'in': {'description': 'dictionary with input parameters', 'required': True, 'type': 'object'}, 'node_class': {'required': True, 'type': 'string'}, 'node_finalized': {'required': True, 'type': 'boolean'}, 'node_history': {'required': True, 'type': 'array'}, 'node_id': {'required': True, 'type': 'integer'}, 'node_name': {'required': True, 'type': 'string'}, 'node_status': {'required': True, 'type': 'string'}, 'out': {'description': 'dictionary with the output results', 'required': True, 'type': 'object'}}, 'type': 'object'}

classmethod from_node(node)

Initialize an instance of NodeResults from a Node subclass.

property exceptions

property gridfs_files

List with the absolute paths of the files to be put in GridFs.

register_gridfs_files(**kwargs)

This function registers the files that will be saved in GridFS. kwargs is a dictionary mapping the key associated to the file (usually the extension) to the absolute path. By default, files are assumed to be in binary form, for formatted files one should pass a tuple (“filepath”, “t”).

Example:

results.register_gridfs(GSR="path/to/GSR.nc", text_file=("/path/to/txt_file", "t"))

The GSR file is a binary file, whereas text_file is a text file.

push_exceptions(*exceptions)

as_dict()

A JSON serializable dict representation of an object.

classmethod from_dict(d)

Parameters

d – Dict representation.

Returns

MSONable class.

json_dump(filename)

classmethod json_load(filename)

validate_json_schema()

update_collection(collection)

Update a mongodb collection.

abipy.flowtk.nodes.check_spectator(node_method)

Decorator for abipy.flowtk.nodes.Node methods. Raise SpectatorNodeError.

exception abipy.flowtk.nodes.NodeError

Bases: Exception

Base Exception raised by abipy.flowtk.nodes.Node subclasses

exception abipy.flowtk.nodes.SpectatorNodeError

Bases: abipy.flowtk.nodes.NodeError

Exception raised by abipy.flowtk.nodes.Node methods when the node is in spectator mode and we are calling a method with side effects.

class abipy.flowtk.nodes.Node

Bases: object

Abstract base class defining the interface that must be implemented by the nodes of the calculation.

Nodes are hashable and can be tested for equality

Results

alias of abipy.flowtk.nodes.NodeResults

Error

alias of abipy.flowtk.nodes.NodeError

SpectatorError

alias of abipy.flowtk.nodes.SpectatorNodeError

S_INIT = <Status: Initialized, at 140630277787264>

S_LOCKED = <Status: Locked, at 140630277787328>

S_READY = <Status: Ready, at 140630277787392>

S_SUB = <Status: Submitted, at 140630277787456>

S_RUN = <Status: Running, at 140630277787520>

S_DONE = <Status: Done, at 140630277787584>

S_ABICRITICAL = <Status: AbiCritical, at 140630277312576>

S_QCRITICAL = <Status: QCritical, at 140630277312640>

S_UNCONVERGED = <Status: Unconverged, at 140630277312704>

S_ERROR = <Status: Error, at 140630277312768>

S_OK = <Status: Completed, at 140630277312832>

ALL_STATUS = [<Status: Initialized, at 140630277787264>, <Status: Locked, at 140630277787328>, <Status: Ready, at 140630277787392>, <Status: Submitted, at 140630277787456>, <Status: Running, at 140630277787520>, <Status: Done, at 140630277787584>, <Status: AbiCritical, at 140630277312576>, <Status: QCritical, at 140630277312640>, <Status: Unconverged, at 140630277312704>, <Status: Error, at 140630277312768>, <Status: Completed, at 140630277312832>]

color_rgb = array([0.41176471, 0.41176471, 0.41176471])

color_hex()

Node color as Hex Triplet https://en.wikipedia.org/wiki/Web_colors#Hex_triplet

isinstance(class_or_string)

Check whether the node is a instance of class_or_string. Unlinke the standard isinstance builtin, the method accepts either a class or a string. In the later case, the string is compared with self.__class__.__name__ (case insensitive).

classmethod as_node(obj)

Convert obj into a Node instance.

Returns

obj if obj is a Node instance, cast obj to FileNode instance of obj is a string. None if obj is None

property name

The name of the node (only used for facilitating its identification in the user interface).

property relworkdir

Return a relative version of the workdir

set_name(name)

Set the name of the Node.

property node_id

Node identifier.

set_readme(md_string)

Set the value of readme_md.

set_abipy_meta_json(data)

Set the value of abipy_meta_json

set_node_id(node_id)

Set the node identifier. Use it carefully!

property finalized

True if the Node has been finalized.

property in_spectator_mode

property corrections

List of dictionaries with infornation on the actions performed to solve AbiCritical Events. Each dictionary contains the AbinitEvent who triggered the correction and a human-readable message with the description of the operation performed.

property num_corrections

log_correction(event, action)

This method should be called once we have fixed the problem associated to this event. It adds a new entry in the correction history of the node.

Parameters

• event – AbinitEvent that triggered the correction.

• action (str) – Human-readable string with info on the action perfomed to solve the problem.

property is_file

True if this node is a file

property is_task

True if this node is a Task

property is_work

True if this node is a Work

property is_flow

True if this node is a Flow

property deps

List of Dependency objects defining the dependencies of this Node. Empty list if this abipy.flowtk.nodes.Node does not have dependencies.

add_deps(deps)

Add a list of dependencies to the abipy.flowtk.nodes.Node.

Parameters

deps – List of Dependency objects specifying the dependencies of the node. or dictionary mapping nodes to file extensions e.g. {task: “DEN”}

merge_deps()

Group all extensions associated to the same node in a single list. Useful for cases in which we may end up with the same node appearing more than once in self.deps. See e.g. add_deps.

remove_deps(deps)

Remove a list of dependencies from the abipy.flowtk.nodes.Node.

Parameters

deps – List of Dependency objects specifying the dependencies of the node.

property deps_status

Returns a list with the status of the dependencies.

depends_on(other)

True if this node depends on the other node.

find_parent_with_ext(ext)

Return the parent (usually a abipy.flowtk.tasks.Task) that produces the file with extension ext. Raises ValueError if multiple parents are found. Return None if no parent is found.

get_parents()

Return the list of nodes in the abipy.flowtk.flows.Flow required by this abipy.flowtk.nodes.Node

get_children()

Return the list of nodes in the abipy.flowtk.flows.Flow that depends on this abipy.flowtk.nodes.Node

Note

This routine assumes the entire flow has been allocated.

str_deps()

Return the string representation of the dependencies of the node.

get_vars_dataframe(*varnames)

Return pandas DataFrame with the value of the variables specified in varnames. Can be used for task/works/flow. It’s recursive!

get_graphviz_dirtree(engine='automatic', **kwargs)

Generate directory graph in the DOT language. The graph show the files and directories in the node workdir.

Returns: graphviz.Digraph <https://graphviz.readthedocs.io/en/stable/api.html#digraph>

set_gc(gc)

Set the garbage collector.

property gc

Garbage collector. None if garbage collection is deactivated. Use flow.set_garbage_collector to initialize the object.

property event_handlers

The list of handlers registered for this node. If the node is not a Flow and does not have its own list of handlers the handlers registered at the level of the flow are returned.

This trick allows one to registered different handlers at the level of the Task for testing purposes. By default, we have a common list of handlers for all the nodes in the flow. This choice facilitates the automatic installation of the handlers when we use callbacks to generate new Works and Tasks!

install_event_handlers(categories=None, handlers=None)

Install the EventHandlers for this `Node. If no argument is provided the default list of handlers is installed.

Parameters

• categories – List of categories to install e.g. base + can_change_physics

• handlers – explicit list of EventHandler instances. This is the most flexible way to install handlers.

Note

categories and handlers are mutually exclusive.

show_event_handlers(stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, verbose=0)

Print to stream the event handlers installed for this flow.

send_signal(signal)

Send signal from this node to all connected receivers unless the node is in spectator mode.

signal – (hashable) signal value, see dispatcher connect for details

Return a list of tuple pairs [(receiver, response), … ] or None if the node is in spectator mode.

if any receiver raises an error, the error propagates back through send, terminating the dispatch loop, so it is quite possible to not have all receivers called if a raises an error.

write_json_in_outdir(filename, data)

Write data to json file of basename filename inside the outdir directory of the node. Support MSONable objects. Return path of json file.

write_json_in_workdir(filename, data)

Write data to json file of basename filename inside the outdir directory of the node. Support MSONable objects. Return path of json file.

abstract property status

The status of the Node.

abstract check_status()

Check the status of the Node.

class abipy.flowtk.nodes.FileNode(filename)

Bases: abipy.flowtk.nodes.Node

A Node that consists of a file. May be not yet existing

Mainly used to connect abipy.flowtk.tasks.Task objects to external files produced in previous runs.

color_rgb = array([0.4, 0.2, 1. ])

basename()

Basename of the file.

property products

opath_from_ext(ext)

property status

The status of the Node.

check_status()

Check the status of the Node.

get_results(**kwargs)

add_filechild(node)

Add a node (usually Task) to the children of this FileNode.

property filechildren

List with the children (nodes) of this FileNode.

abiopen()

open_gsr()

class abipy.flowtk.nodes.HistoryRecord(level, pathname, lineno, msg, args, exc_info, func=None)

Bases: object

A HistoryRecord instance represents an entry in the NodeHistory.

HistoryRecord instances are created every time something is logged. They contain all the information pertinent to the event being logged. The main information passed in is in msg and args, which are combined using str(msg) % args to create the message field of the record. The record also includes information such as when the record was created, the source line where the logging call was made

levelno

Numeric logging level for the message (DEBUG, INFO, WARNING, ERROR, CRITICAL)

levelname

Text logging level for the message (“DEBUG”, “INFO”, “WARNING”, “ERROR”, “CRITICAL”)

pathname

Full pathname of the source file where the logging call was issued (if available)

filename

Filename portion of pathname

module

Module (name portion of filename)

lineno

Source line number where the logging call was issued (if available)

func_name

Function name

created

Time when the HistoryRecord was created (time.time() return value)

asctime

Textual time when the HistoryRecord was created

message

The result of record.getMessage(), computed just as the record is emitted

get_message(metadata=False, asctime=True)

Return the message after merging any user-supplied arguments with the message.

Parameters

• metadata – True if function and module name should be added.

• asctime – True if time string should be added.

as_dict()

classmethod from_dict(d)

class abipy.flowtk.nodes.NodeHistory

Bases: collections.deque

Logger-like object

to_string(metadata=False)

Returns a string with the history. Set metadata to True to have info on function and module.

info(msg, *args, **kwargs)

Log ‘msg % args’ with the info severity level

warning(msg, *args, **kwargs)

Log ‘msg % args’ with the warning severity level

critical(msg, *args, **kwargs)

Log ‘msg % args’ with the critical severity level

debug(msg, *args, **kwargs)

Log ‘msg % args’ with the critical severity level

class abipy.flowtk.nodes.NodeCorrections(iterable=(), /)

Bases: list

Iterable storing the correctios performed by the EventHandler

class abipy.flowtk.nodes.GarbageCollector(exts, policy)

Bases: object

This object stores information on the

abipy.flowtk.nodes.init_counter()

abipy.flowtk.nodes.get_newnode_id()

Returns a new node identifier used for abipy.flowtk.tasks.Task, abipy.flowtk.works.Work and abipy.flowtk.flows.Flow objects.

abipy.flowtk.nodes.save_lastnode_id()

Save the id of the last node created.

pseudos Module

qadapters Module

The initial version of this module was based on a similar implementation present in FireWorks (https://pypi.python.org/pypi/FireWorks). Work done by D. Waroquiers, A. Jain, and M. Kocher.

The main difference wrt the Fireworks implementation is that the QueueAdapter objects provide a programmatic interface for setting important attributes such as the number of MPI nodes, the number of OMP threads and the memory requirements. This programmatic interface is used by the TaskManager for optimizing the parameters of the run before submitting the job (Abinit provides the autoparal option that allows one to get a list of parallel configuration and their expected efficiency).

abipy.flowtk.qadapters.make_qadapter(**kwargs)

Return the concrete QueueAdapter class from a string. Note that one can register a customized version with:

from qadapters import SlurmAdapter

class MyAdapter(SlurmAdapter):
    QTYPE = "myslurm"
    # Add your customized code here

# Register your class.
SlurmAdapter.register(MyAdapter)

make_qadapter(qtype="myslurm", **kwargs)

Warning

MyAdapter should be pickleable, hence one should declare it at the module level so that pickle can import it at run-time.

qha Module

Workflows for calculations within the quasi-harmonic approximation.

class abipy.flowtk.qha.RelaxAndPhononWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

This work performs a structural relaxation for different volumes, then it uses the relaxed structures to compute phonons, BECS and the dielectric tensor with DFPT.

Inheritance Diagram

classmethod from_scf_input(scf_input, volumes, ngqpt, with_becs, optcell, ionmov, edos_ngkpt=None)

Build the work from an abipy.abio.inputs.AbinitInput representing a GS-SCF calculation.

Parameters

• scf_input – abipy.abio.inputs.AbinitInput for GS-SCF used as template to generate the other inputs.

• volumes – List of volumes in Ang**3

• with_becs – Activate calculation of Electric field and Born effective charges.

• optcell – Abinit input variables.

• ionmov – Abinit input variables.

• edos_ngkpt – Three integers defining the the k-sampling for the computation of the electron DOS with the relaxed structures. Useful for metals or small gap semiconductors in which the electronic contribution should be included. None disables the computation of the e-DOS.

on_all_ok()

This callback is called when all tasks in the Work reach status S_OK. Here are add a new PhononWork for each volume using the relaxed structure.

class abipy.flowtk.qha.QhaFlow(workdir, manager=None, pickle_protocol=- 1, remove=False)

Bases: abipy.flowtk.flows.Flow

Inheritance Diagram

classmethod from_scf_input(workdir, scf_input, volumes, ngqpt, with_becs, edos_ngkpt=None, metadata=None, manager=None)

Build a abipy.flowtk.flows.Flow for QHA calculations from an abipy.abio.inputs.AbinitInput for GS-SCF calculations.

Parameters

• workdir – Working directory of the flow.

• scf_input – abipy.abio.inputs.AbinitInput for GS-SCF run used as template to generate the other inputs.

• volumes – List of volumes in Ang**3.

• ngqpt – Three integers defining the q-mesh for phonon calculation.

• with_becs – Activate calculation of Electric field and Born effective charges.

• edos_ngkpt – Three integers defining the the k-sampling for the computation of the electron DOS with the relaxed structures. Useful for metals or small gap semiconductors in which the electronic contribution should be included. None disables the computation of the e-DOS.

• metadata – Dictionary with metadata to be be added to the final JSON file.

• manager – abipy.flowtk.tasks.TaskManager instance. Use default if None.

finalize()

This method is called when the flow is completed. It performs some basic post-processing of the results to facilitate further analysis.

qjobs Module

Objects and methods to contact the resource manager to get info on the status of the job and useful statistics. Note that this is not a wrapper for the C API but a collection of simple wrappers around the shell commands provided by the resource manager (qsub, qdel and qstat for PBS, sinfo, squeue… for Slurm). The main goal indeed is providing a simplified common interface for different resource managers without having to rely on external libraries.

class abipy.flowtk.qjobs.JobStatus

Bases: int

This object is an integer representing the status of a QueueJob.

Slurm API, see man squeue.

JOB STATE CODES

Jobs typically pass through several states in the course of their execution. The typical states are PENDING, RUNNING, SUSPENDED, COMPLETING, and COMPLETED. An explanation of each state follows:

BF  BOOT_FAIL       Job  terminated  due  to launch failure, typically due to a hardware failure (e.g.
                    unable to boot the node or block and the job can not be requeued).
CA  CANCELLED       Job was explicitly cancelled by the user or system administrator.
                    The job may or may not have been initiated.
CD  COMPLETED       Job has terminated all processes on all nodes.
CF  CONFIGURING     Job has been allocated resources, but are waiting for them to become ready for use (e.g. booting).
CG  COMPLETING      Job is in the process of completing. Some processes on some  nodes may still be active.
F   FAILED          Job terminated with non-zero exit code or other failure condition.
NF  NODE_FAIL       Job terminated due to failure of one or more allocated nodes.
PD  PENDING         Job is awaiting resource allocation.
PR  PREEMPTED       Job terminated due to preemption.
R   RUNNING         Job currently has an allocation.
S   SUSPENDED       Job has an allocation, but execution has been suspended.
TO  TIMEOUT         Job terminated upon reaching its time limit.
SE SPECIAL_EXIT     The job was requeued in a special state. This state can be set by users, typically
                    in EpilogSlurmctld, if the job has terminated with a particular exit value.

classmethod from_string(s)

Return a JobStatus instance from its string representation.

class abipy.flowtk.qjobs.QueueJob(queue_id, qname='UnknownQueue')

Bases: object

This object provides methods to contact the resource manager to get info on the status of the job and useful statistics. This is an abstract class.

QTYPE = None

S_UNKNOWN = <JobStatus: UNKNOWN, at 140630277112320>

S_PENDING = <JobStatus: PENDING, at 140630277112384>

S_RUNNING = <JobStatus: RUNNING, at 140630277112448>

S_RESIZING = <JobStatus: RESIZING, at 140630277112512>

S_SUSPENDED = <JobStatus: SUSPENDED, at 140630277112576>

S_COMPLETED = <JobStatus: COMPLETED, at 140630277112640>

S_CANCELLED = <JobStatus: CANCELLED, at 140630277112704>

S_FAILED = <JobStatus: FAILED, at 140630277112768>

S_TIMEOUT = <JobStatus: TIMEOUT, at 140630277112832>

S_PREEMPTED = <JobStatus: PREEMPTED, at 140630277112896>

S_NODEFAIL = <JobStatus: NODEFAIL, at 140630277112960>

static from_qtype_and_id(qtype, queue_id, qname=None)

Return a new istance of the appropriate subclass.

Parameters

• qtype – String specifying the Resource manager type.

• queue_id – Job identifier.

• qname – Name of the queue (optional).

property is_completed

property is_running

property is_failed

property timeout

property has_node_failures

property unknown_status

set_status_exitcode_signal(status, exitcode, signal)

likely_code_error()

See <http://man7.org/linux/man-pages/man7/signal.7.html>

SIGHUP	1	Term	Hangup detected on controlling terminal or death of controlling process
SIGINT	2	Term	Interrupt from keyboard
SIGQUIT	3	Core	Quit from keyboard
SIGILL	4	Core	Illegal Instruction
SIGABRT	6	Core	Abort signal from abort(3)
SIGFPE	8	Core	Floating point exception
SIGKILL	9	Term	Kill signal
SIGSEGV	11	Core	Invalid memory reference
SIGPIPE	13	Term	Broken pipe: write to pipe with no readers
SIGALRM	14	Term	Timer signal from alarm(2)
SIGTERM	15	Term	Termination signal
SIGUSR1	30,10,16	Term	User-defined signal 1
SIGUSR2	31,12,17	Term	User-defined signal 2
SIGCHLD	20,17,18	Ign	Child stopped or terminated
SIGCONT	19,18,25	Cont	Continue if stopped
SIGSTOP	17,19,23	Stop	Stop process
SIGTSTP	18,20,24	Stop	Stop typed at terminal
SIGTTIN	21,21,26	Stop	Terminal input for background process
SIGTTOU	22,22,27	Stop	Terminal output for background process

The signals SIGKILL and SIGSTOP cannot be caught, blocked, or ignored. Next the signals not in the POSIX.1-1990 standard but described in SUSv2 and POSIX.1-2001.

Signal	Value	Action	Comment
SIGBUS	10,7,10	Core	Bus error (bad memory access)
SIGPOLL		Term	Pollable event (Sys V). Synonym for SIGIO
SIGPROF	27,27,29	Term	Profiling timer expired
SIGSYS	12,31,12	Core	Bad argument to routine (SVr4)
SIGTRAP	5	Core	Trace/breakpoint trap
SIGURG	16,23,21	Ign	Urgent condition on socket (4.2BSD)
SIGVTALRM	26,26,28	Term	Virtual alarm clock (4.2BSD)
SIGXCPU	24,24,30	Core	CPU time limit exceeded (4.2BSD)
SIGXFSZ	25,25,31	Core	File size limit exceeded (4.2BSD)

received_signal(sig_name)

estimated_start_time()

Return date with estimated start time. None if it cannot be detected

get_info(**kwargs)

get_nodes(**kwargs)

get_stats(**kwargs)

class abipy.flowtk.qjobs.ShellJob(queue_id, qname='UnknownQueue')

Bases: abipy.flowtk.qjobs.QueueJob

Handler for Shell jobs.

QTYPE = 'shell'

class abipy.flowtk.qjobs.SlurmJob(queue_id, qname='UnknownQueue')

Bases: abipy.flowtk.qjobs.QueueJob

Handler for Slurm jobs.

QTYPE = 'slurm'

estimated_start_time()

Return date with estimated start time. None if it cannot be detected

get_info(**kwargs)

get_stats(**kwargs)

class abipy.flowtk.qjobs.PbsProJob(queue_id, qname='UnknownQueue')

Bases: abipy.flowtk.qjobs.QueueJob

Handler for PbsPro Jobs.

See also https://github.com/plediii/pbs_util for a similar project.

QTYPE = 'pbspro'

PBSSTAT_TO_SLURM = {'E': <JobStatus: FAILED, at 140630277112768>, 'F': <JobStatus: COMPLETED, at 140630277112640>, 'Q': <JobStatus: PENDING, at 140630277112384>, 'R': <JobStatus: RUNNING, at 140630277112448>, 'S': <JobStatus: SUSPENDED, at 140630277112576>}

estimated_start_time()

Return date with estimated start time. None if it cannot be detected

get_info(**kwargs)

class abipy.flowtk.qjobs.TorqueJob(queue_id, qname='UnknownQueue')

Bases: abipy.flowtk.qjobs.QueueJob

Not supported

QTYPE = 'torque'

class abipy.flowtk.qjobs.SgeJob(queue_id, qname='UnknownQueue')

Bases: abipy.flowtk.qjobs.QueueJob

Not supported

QTYPE = 'sge'

class abipy.flowtk.qjobs.MoabJob(queue_id, qname='UnknownQueue')

Bases: abipy.flowtk.qjobs.QueueJob

Not supported

QTYPE = 'moab'

class abipy.flowtk.qjobs.BlueGeneJob(queue_id, qname='UnknownQueue')

Bases: abipy.flowtk.qjobs.QueueJob

Not supported

QTYPE = 'bluegene'

qutils Module

Collection of low-level tools that faciliate the interface with resource managers.

The preferred way of importing this module is:

import qutils as qu

abipy.flowtk.qutils.slurm_parse_timestr(s)

A slurm time parser. Accepts a string in one the following forms:

# “days-hours”, # “days-hours:minutes”, # “days-hours:minutes:seconds”. # “minutes”, # “minutes:seconds”, # “hours:minutes:seconds”,

Returns: Time in seconds.

Raises

ValueError –

abipy.flowtk.qutils.time2slurm(timeval, unit='s')

Convert a number representing a time value in the given unit (Default: seconds) to a string following the slurm convention: “days-hours:minutes:seconds”.

>>> assert time2slurm(61) == '0-0:1:1' and time2slurm(60*60+1) == '0-1:0:1'
>>> assert time2slurm(0.5, unit="h") == '0-0:30:0'

abipy.flowtk.qutils.time2pbspro(timeval, unit='s')

Convert a number representing a time value in the given unit (Default: seconds) to a string following the PbsPro convention: “hours:minutes:seconds”.

>>> assert time2pbspro(2, unit="d") == '48:0:0'

abipy.flowtk.qutils.time2loadlever(timeval, unit='s')

Convert a number representing a time value in the given unit (Default: seconds) to a string following the LoadLever convention. format hh:mm:ss (hours:minutes:seconds)

>>> assert time2loadlever(2, unit="d") == '48:00:00'

abipy.flowtk.qutils.timelimit_parser(s)

Convert a float or a string into time in seconds.

abipy.flowtk.qutils.any2mb(s)

Convert string or number to memory in megabytes.

tasks Module

This module provides functions and classes related to Task objects.

class abipy.flowtk.tasks.TaskManager(**kwargs)

Bases: monty.json.MSONable

A TaskManager is responsible for the generation of the job script and the submission of the task, as well as for the specification of the parameters passed to the resource manager (e.g. Slurm, PBS …) and/or the run-time specification of the ABINIT variables governing the parallel execution. A TaskManager delegates the generation of the submission script and the submission of the task to the QueueAdapter. A TaskManager has a TaskPolicy that governs the specification of the parameters for the parallel executions. Ideally, the TaskManager should be the main entry point used by the task to deal with job submission/optimization

YAML_FILE = 'manager.yml'

USER_CONFIG_DIR = '/Users/gmatteo/.abinit/abipy'

ENTRIES = {'batch_adapter', 'db_connector', 'policy', 'qadapters'}

classmethod autodoc()

classmethod get_simple_manager()

classmethod from_user_config()

Initialize the abipy.flowtk.tasks.TaskManager from the YAML file ‘manager.yaml’. Search first in the working directory and then in the AbiPy configuration directory.

Raises

RuntimeError if file is not found. –

classmethod from_file(filename)

Read the configuration parameters from the Yaml file filename.

classmethod from_string(s)

Create an instance from string s containing a YAML dictionary.

classmethod as_manager(obj)

Convert obj into TaskManager instance. Accepts string, filepath, dictionary, TaskManager object. If obj is None, the manager is initialized from the user config file.

classmethod from_dict(d)

Create an instance from a dictionary.

as_dict()

A JSON serializable dict representation of an object.

abinit_build()

AbinitBuild object with Abinit version and options used to build the code

to_shell_manager(mpi_procs=1)

Returns a new abipy.flowtk.tasks.TaskManager with the same parameters as self but replace the QueueAdapter with a ShellAdapter with mpi_procs so that we can submit the job without passing through the queue.

new_with_fixed_mpi_omp(mpi_procs, omp_threads)

Return a new TaskManager in which autoparal has been disabled. The jobs will be executed with mpi_procs MPI processes and omp_threads OpenMP threads. Useful for generating input files for benchmarks.

property has_queue

True if we are submitting jobs via a queue manager.

property qads

List of QueueAdapter objects sorted according to priorities (highest comes first)

property qadapter

The qadapter used to submit jobs.

select_qadapter(pconfs)

Given a list of parallel configurations, pconfs, this method select an optimal configuration according to some criterion as well as the QueueAdapter to use.

Parameters

pconfs – ParalHints object with the list of parallel configurations

Returns

ParallelConf object with the optimal configuration.

property has_db

True if we are using MongoDB database

property has_omp

True if we are using OpenMP parallelization.

property num_cores

Total number of CPUs used to run the task.

property mpi_procs

Number of MPI processes.

property mem_per_proc

Memory per MPI process.

property omp_threads

Number of OpenMP threads

deepcopy()

Deep copy of self.

set_mpi_procs(mpi_procs)

Set the number of MPI processes to use.

set_omp_threads(omp_threads)

Set the number of OpenMp threads to use.

set_mem_per_proc(mem_mb)

Set the memory (in Megabytes) per CPU.

property max_cores

Maximum number of cores that can be used. This value is mainly used in the autoparal part to get the list of possible configurations.

get_njobs_in_queue(username=None)

returns the number of jobs in the queue, returns None when the number of jobs cannot be determined.

Parameters

username – (str) the username of the jobs to count (default is to autodetect)

cancel(job_id)

Cancel the job. Returns exit status.

write_jobfile(task, **kwargs)

Write the submission script. Return the path of the script

kwargs	Meaning
exec_args	List of arguments passed to task.executable. Default: no arguments.

launch(task, **kwargs)

Build the input files and submit the task via the Qadapter

Parameters

task – abipy.flowtk.tasks.Task object.

Returns

Process object.

get_collection(**kwargs)

Return the MongoDB collection used to store the results.

increase_mem()

increase_ncpus()

increase the number of cpus, first ask the current qadapter, if that one raises a QadapterIncreaseError switch to the next qadapter. If all fail raise an ManagerIncreaseError

increase_resources()

exclude_nodes(nodes)

increase_time()

class abipy.flowtk.tasks.AbinitBuild(workdir=None, manager=None)

Bases: object

This object stores information on the options used to build Abinit

info

String with build information as produced by `abinit -b`

version

Abinit version number e.g 8.0.1 (string)

has_netcdf

True if netcdf is enabled.

has_libxc

True if libxc is enabled.

has_omp

True if OpenMP is enabled.

has_mpi

True if MPI is enabled.

has_mpiio

True if MPI-IO is supported.

version_ge(version_string)

True is Abinit version is >= version_string

compare_version(version_string, op)

Compare Abinit version to version_string with operator op

class abipy.flowtk.tasks.ParalHintsParser

Bases: object

Error

alias of abipy.flowtk.tasks.ParalHintsError

add_error(errmsg)

parse(filename)

Read the AutoParal section (YAML format) from filename. Assumes the file contains only one section.

class abipy.flowtk.tasks.ParalHints(info, confs)

Bases: collections.abc.Iterable

Iterable with the hints for the parallel execution reported by ABINIT.

Error

alias of abipy.flowtk.tasks.ParalHintsError

classmethod from_mpi_omp_lists(mpi_procs, omp_threads)

Build a list of Parallel configurations from two lists containing the number of MPI processes and the number of OpenMP threads i.e. product(mpi_procs, omp_threads). The configuration have parallel efficiency set to 1.0 and no input variables. Mainly used for preparing benchmarks.

max_cores()

Maximum number of cores.

max_mem_per_proc()

Maximum memory per MPI process.

max_speedup()

Maximum speedup.

max_efficiency()

Maximum parallel efficiency.

as_dict(**kwargs)

classmethod from_dict(d)

copy()

Shallow copy of self.

select_with_condition(condition, key=None)

Remove all the configurations that do not satisfy the given condition.

Parameters

• condition – dict or Condition object with operators expressed with a Mongodb-like syntax

• key – Selects the sub-dictionary on which condition is applied, e.g. key=”vars” if we have to filter the configurations depending on the values in vars

sort_by_efficiency(reverse=True)

Sort the configurations in place. items with highest efficiency come first

sort_by_speedup(reverse=True)

Sort the configurations in place. items with highest speedup come first

sort_by_mem_per_proc(reverse=False)

Sort the configurations in place. items with lowest memory per proc come first.

multidimensional_optimization(priorities=('speedup', 'efficiency'))

get_ordered_with_policy(policy, max_ncpus)

Sort and return a new list of configurations ordered according to the TaskPolicy policy.

class abipy.flowtk.tasks.AbinitTask(input, workdir=None, manager=None, deps=None)

Bases: abipy.flowtk.tasks.Task

Base class defining an ABINIT calculation

Results

alias of abipy.flowtk.tasks.TaskResults

classmethod from_input(input, workdir=None, manager=None)

Create an instance of AbinitTask from an ABINIT input.

Parameters

• ainput – abipy.abio.inputs.AbinitInput object.

• workdir – Path to the working directory.

• manager – abipy.flowtk.tasks.TaskManager object.

classmethod temp_shell_task(inp, mpi_procs=1, workdir=None, manager=None)

Build a Task with a temporary workdir. The task is executed via the shell with 1 MPI proc. Mainly used for invoking Abinit to get important parameters needed to prepare the real task.

Parameters

mpi_procs – Number of MPI processes to use.

setup()

Abinit has the very bad habit of changing the file extension by appending the characters in [A,B …, Z] to the output file, and this breaks a lot of code that relies of the use of a unique file extension. Here we fix this issue by renaming run.abo to run.abo_[number] if the output file “run.abo” already exists. A few lines of code in python, a lot of problems if you try to implement this trick in Fortran90.

property executable

Path to the executable required for running the Task.

property pseudos

List of pseudos used in the calculation.

property isnc

True if norm-conserving calculation.

property ispaw

True if PAW calculation

property is_gs_task

True if task is GsTask subclass.

property is_dfpt_task

True if task is a DftpTask subclass.

cycle_class()

Return the subclass of ScfCycle associated to the task or None if no SCF algorithm if associated to the task.

property filesfile_string

String with the list of files and prefixes needed to execute ABINIT.

set_pconfs(pconfs)

Set the list of autoparal configurations.

property pconfs

List of autoparal configurations.

uses_paral_kgb(value=1)

True if the task is a GS Task and uses paral_kgb with the given value.

get_panel(**kwargs)

Build panel with widgets to interact with the Task either in a notebook or in panel app. This is the implementation provided by the base class. Subclasses may provide specialized implementations.

autoparal_run()

Find an optimal set of parameters for the execution of the task This method can change the ABINIT input variables and/or the submission parameters e.g. the number of CPUs for MPI and OpenMp.

Set:

self.pconfs where pconfs is a ParalHints object with the configuration reported by autoparal and optimal is the optimal configuration selected. Returns 0 if success

find_optconf(pconfs)

Find the optimal Parallel configuration.

select_files(what='o')

Helper function used to select the files of a task.

Parameters

what – string with the list of characters selecting the file type Possible choices: i ==> input_file, o ==> output_file, f ==> files_file, j ==> job_file, l ==> log_file, e ==> stderr_file, q ==> qout_file, all ==> all files.

restart()

general restart used when scheduler problems have been taken care of

reset_from_scratch()

Restart from scratch, this is to be used if a job is restarted with more resources after a crash

Move output files produced in workdir to _reset otherwise check_status continues to see the task as crashed even if the job did not run

fix_abicritical()

method to fix crashes/error caused by abinit

Returns

1 if task has been fixed else 0.

fix_queue_critical()

This function tries to fix critical events originating from the queue submission system.

General strategy, first try to increase resources in order to fix the problem, if this is not possible, call a task specific method to attempt to decrease the demands.

Returns

1 if task has been fixed else 0.

parse_timing()

Parse the timer data in the main output file of Abinit. Requires timopt /= 0 in the input file (usually timopt = -1)

Return: AbinitTimerParser instance, None if error.

class abipy.flowtk.tasks.ScfTask(input, workdir=None, manager=None, deps=None)

Bases: abipy.flowtk.tasks.GsTask

Self-consistent ground-state calculations. Provide support for in-place restart via (WFK|DEN) files

CRITICAL_EVENTS = [<class 'abipy.flowtk.events.ScfConvergenceWarning'>]

color_rgb = array([1., 0., 0.])

restart()

SCF calculations can be restarted if we have either the WFK file or the DEN file.

inspect(**kwargs)

Plot the SCF cycle results with matplotlib.

Returns: matplotlib.figure.Figure or None if some error occurred.

get_results(**kwargs)

Returns NodeResults instance. Subclasses should extend this method (if needed) by adding specialized code that performs some kind of post-processing.

add_ebands_task_to_work(work, ndivsm=15, tolwfr=1e-20, nscf_nband=None, nb_extra=10)

Generate an NSCF task form band structure calculation from a GS SCF task and add it to the work.

Parameters

• work –

• ndivsm – if > 0, it’s the number of divisions for the smallest segment of the path (Abinit variable). if < 0, it’s interpreted as the pymatgen line_density parameter in which the number of points in the segment is proportional to its length. Typical value: -20. This option is the recommended one if the k-path contains two high symmetry k-points that are very close as ndivsm > 0 may produce a very large number of wavevectors.

• tolwfr – Tolerance on residuals for NSCF calculation.

• nscf_nband – Number of bands for NSCF calculation. If None, use nband + nb_extra

return: NscfTask object.

class abipy.flowtk.tasks.NscfTask(input, workdir=None, manager=None, deps=None)

Bases: abipy.flowtk.tasks.GsTask

Non-Self-consistent GS calculation. Provide in-place restart via WFK files

CRITICAL_EVENTS = [<class 'abipy.flowtk.events.NscfConvergenceWarning'>]

color_rgb = array([0.78431373, 0.31372549, 0.39215686])

setup()

NSCF calculations should use the same FFT mesh as the one employed in the GS task (in principle, it’s possible to interpolate inside Abinit but tests revealed some numerical noise Here we change the input file of the NSCF task to have the same FFT mesh.

restart()

NSCF calculations can be restarted only if we have the WFK file.

get_results(**kwargs)

Returns NodeResults instance. Subclasses should extend this method (if needed) by adding specialized code that performs some kind of post-processing.

class abipy.flowtk.tasks.RelaxTask(input, workdir=None, manager=None, deps=None)

Bases: abipy.flowtk.tasks.GsTask, abipy.flowtk.tasks.ProduceHist

Task for structural optimizations.

CRITICAL_EVENTS = [<class 'abipy.flowtk.events.RelaxConvergenceWarning'>]

color_rgb = array([1.        , 0.23921569, 1.        ])

get_final_structure()

Read the final structure from the GSR file.

restart()

Restart the structural relaxation.

Structure relaxations can be restarted only if we have the WFK file or the DEN or the GSR file from which we can read the last structure (mandatory) and the wavefunctions (not mandatory but useful). Prefer WFK over other files since we can reuse the wavefunctions.

Note

The problem in the present approach is that some parameters in the input are computed from the initial structure and may not be consistent with the modification of the structure done during the structure relaxation.

inspect(**kwargs)

Plot the evolution of the structural relaxation with matplotlib.

Parameters

what – Either “hist” or “scf”. The first option (default) extracts data from the HIST file and plot the evolution of the structural parameters, forces, pressures and energies. The second option, extracts data from the main output file and plot the evolution of the SCF cycles (etotal, residuals, etc).

Returns: matplotlib.figure.Figure or None if some error occurred.

get_results(**kwargs)

Returns NodeResults instance. Subclasses should extend this method (if needed) by adding specialized code that performs some kind of post-processing.

reduce_dilatmx(target=1.01)

fix_ofiles()

Note that ABINIT produces lots of out_TIM1_DEN files for each step. Here we list all TIM*_DEN files, we select the last one and we rename it as out_DEN

This change is needed so that we can specify dependencies with the syntax {node: “DEN”} without having to know the number of iterations needed to converge.

class abipy.flowtk.tasks.DdkTask(input, workdir=None, manager=None, deps=None)

Bases: abipy.flowtk.tasks.DfptTask

Task for DDK calculations.

color_rgb = array([0. , 0.8, 0.8])

get_results(**kwargs)

Returns NodeResults instance. Subclasses should extend this method (if needed) by adding specialized code that performs some kind of post-processing.

class abipy.flowtk.tasks.EffMassTask(input, workdir=None, manager=None, deps=None)

Bases: abipy.flowtk.tasks.DfptTask

Task for effective mass calculations with DFPT.

color_rgb = array([0.        , 0.47843137, 0.8       ])

class abipy.flowtk.tasks.PhononTask(input, workdir=None, manager=None, deps=None)

Bases: abipy.flowtk.tasks.DfptTask

DFPT calculations for a single atomic perturbation. Provide support for in-place restart via (1WF|1DEN) files

color_rgb = array([0.        , 0.58823529, 0.98039216])

inspect(**kwargs)

Plot the Phonon SCF cycle results with matplotlib.

Returns: matplotlib.figure.Figure or None if some error occurred.

get_results(**kwargs)

Returns NodeResults instance. Subclasses should extend this method (if needed) by adding specialized code that performs some kind of post-processing.

class abipy.flowtk.tasks.ElasticTask(input, workdir=None, manager=None, deps=None)

Bases: abipy.flowtk.tasks.DfptTask

DFPT calculations for a single strain perturbation (uniaxial or shear strain). Provide support for in-place restart via (1WF|1DEN) files

color_rgb = array([1. , 0.8, 1. ])

class abipy.flowtk.tasks.SigmaTask(input, workdir=None, manager=None, deps=None)

Bases: abipy.flowtk.tasks.ManyBodyTask

Tasks for SIGMA calculations. Provides support for in-place restart via QPS files

CRITICAL_EVENTS = [<class 'abipy.flowtk.events.QPSConvergenceWarning'>]

color_rgb = array([0., 1., 0.])

restart()

general restart used when scheduler problems have been taken care of

property sigres_path

Absolute path of the SIGRES file. Empty string if file is not present.

open_sigres()

Open the SIGRES file located in the in self.outdir. Returns abipy.electrons.gw.SigresFile object, None if file could not be found or file is not readable.

get_scissors_builder()

Returns an instance of ScissorsBuilder from the SIGRES file.

Raises

RuntimeError –

get_results(**kwargs)

Returns NodeResults instance. Subclasses should extend this method (if needed) by adding specialized code that performs some kind of post-processing.

class abipy.flowtk.tasks.EphTask(input, workdir=None, manager=None, deps=None)

Bases: abipy.flowtk.tasks.AbinitTask

Class for electron-phonon calculations.

color_rgb = array([1.        , 0.50196078, 0.        ])

class abipy.flowtk.tasks.KerangeTask(input, workdir=None, manager=None, deps=None)

Bases: abipy.flowtk.tasks.AbinitTask

Class for kerange calculations.

color_rgb = array([1.        , 0.50196078, 0.50196078])

class abipy.flowtk.tasks.OpticTask(optic_input, nscf_node, ddk_nodes, use_ddknc=False, workdir=None, manager=None)

Bases: abipy.flowtk.tasks.Task

Task for the computation of optical spectra with optic i.e. RPA without local-field effects and velocity operator computed from DDK files.

color_rgb = array([1. , 0.8, 0.4])

set_workdir(workdir, chroot=False)

Set the working directory of the task.

set_vars(*args, **kwargs)

Optic does not use get or ird variables hence we should never try to change the input when we connect this task

property executable

Path to the executable required for running the OpticTask.

property filesfile_string

String with the list of files and prefixes needed to execute ABINIT.

property wfk_filepath

Returns (at runtime) the absolute path of the WFK file produced by the NSCF run.

property ddk_filepaths

Returns (at runtime) the absolute path of the DDK files produced by the DDK runs.

make_input()

Construct and write the input file of the calculation.

setup()

Public method called before submitting the task.

make_links()

Optic allows the user to specify the paths of the input file. hence we don’t need to create symbolic links.

get_results(**kwargs)

Returns NodeResults instance. Subclasses should extend this method (if needed) by adding specialized code that performs some kind of post-processing.

fix_abicritical()

Cannot fix abicritical errors for optic

reset_from_scratch()

restart from scratch, this is to be used if a job is restarted with more resources after a crash

fix_queue_critical()

This function tries to fix critical events originating from the queue submission system.

General strategy, first try to increase resources in order to fix the problem, if this is not possible, call a task specific method to attempt to decrease the demands.

Returns

1 if task has been fixed else 0.

autoparal_run()

Find an optimal set of parameters for the execution of the Optic task This method can change the submission parameters e.g. the number of CPUs for MPI and OpenMp.

Returns 0 if success

class abipy.flowtk.tasks.AnaddbTask(anaddb_input, ddb_node, gkk_node=None, md_node=None, ddk_node=None, workdir=None, manager=None)

Bases: abipy.flowtk.tasks.Task

Task for Anaddb runs (post-processing of DFPT calculations).

color_rgb = array([0.8, 0.4, 1. ])

classmethod temp_shell_task(inp, ddb_node, mpi_procs=1, gkk_node=None, md_node=None, ddk_node=None, workdir=None, manager=None)

Build a abipy.flowtk.tasks.AnaddbTask with a temporary workdir. The task is executed via the shell with 1 MPI proc. Mainly used for post-processing the DDB files.

Parameters

• mpi_procs – Number of MPI processes to use.

• anaddb_input – string with the anaddb variables.

• ddb_node – The node that will produce the DDB file. Accept abipy.flowtk.tasks.Task, abipy.flowtk.works.Work or filepath.

See AnaddbInit for the meaning of the other arguments.

property executable

Path to the executable required for running the abipy.flowtk.tasks.AnaddbTask.

property filesfile_string

String with the list of files and prefixes needed to execute ABINIT.

property ddb_filepath

Returns (at runtime) the absolute path of the input DDB file.

property md_filepath

Returns (at runtime) the absolute path of the input MD file.

property gkk_filepath

Returns (at runtime) the absolute path of the input GKK file.

property ddk_filepath

Returns (at runtime) the absolute path of the input DKK file.

setup()

Public method called before submitting the task.

make_links()

Anaddb allows the user to specify the paths of the input file. hence we don’t need to create symbolic links.

outpath_from_ext(ext)

open_phbst()

Open PHBST file produced by Anaddb and returns abipy.dfpt.phonons.PhbstFile object.

open_phdos()

Open PHDOS file produced by Anaddb and returns abipy.dfpt.phonons.PhdosFile object.

get_results(**kwargs)

Returns NodeResults instance. Subclasses should extend this method (if needed) by adding specialized code that performs some kind of post-processing.

abipy.flowtk.tasks.set_user_config_taskmanager(manager)

Change the default manager returned by TaskManager.from_user_config.

utils Module

Tools and helper functions for abinit calculations

abipy.flowtk.utils.as_bool(s)

Convert a string into a boolean.

>>> assert as_bool(True) is True and as_bool("Yes") is True and as_bool("false") is False

class abipy.flowtk.utils.File(path)

Bases: object

Very simple class used to store file basenames, absolute paths and directory names. Provides wrappers for the most commonly used functions defined in os.path.

property path

Absolute path of the file.

property basename

File basename.

property relpath

Relative path.

property dirname

Absolute path of the directory where the file is located.

property exists

True if file exists.

property isncfile

True if self is a NetCDF file

chmod(mode)

Change the access permissions of a file.

read()

Read data from file.

readlines()

Read lines from files.

write(string)

Write string to file.

writelines(lines)

Write a list of strings to file.

make_dir()

Make the directory where the file is located.

remove()

Remove the file.

move(dst)

Recursively move a file or directory to another location. This is similar to the Unix “mv” command.

get_stat()

Results from os.stat

getsize()

Return the size, in bytes, of path. Return 0 if the file is empty or it does not exist.

class abipy.flowtk.utils.Directory(path)

Bases: object

Very simple class that provides helper functions wrapping the most commonly used functions defined in os.path.

property path

Absolute path of the directory.

property relpath

Relative path.

property basename

Directory basename.

path_join(*p)

Join two or more pathname components, inserting ‘/’ as needed. If any component is an absolute path, all previous path components will be discarded.

property exists

True if file exists.

makedirs()

Super-mkdir; create a leaf directory and all intermediate ones. Works like mkdir, except that any intermediate path segment (not just the rightmost) will be created if it does not exist.

rmtree()

Recursively delete the directory tree

copy_r(dst)

Implements a recursive copy function similar to Unix’s “cp -r” command.

clean()

Remove all files in the directory tree while preserving the directory

path_in(file_basename)

Return the absolute path of filename in the directory.

list_filepaths(wildcard=None)

Return the list of absolute filepaths in the directory.

Parameters

wildcard –

String of tokens separated by “|”. Each token represents a pattern. If wildcard is not None, we return only those files whose basename matches the given shell pattern (uses fnmatch). .. rubric:: Example

wildcard=”.nc|.pdf” selects only those files that end with .nc or .pdf

has_abiext(ext, single_file=True)

Returns the absolute path of the ABINIT file with extension ext. Support both Fortran files and netcdf files. In the later case, we check whether a file with extension ext + “.nc” is present in the directory. Returns empty string is file is not present.

Raises

• ValueError –

• This implies that this method is not compatible with multiple datasets. –

symlink_abiext(inext, outext)

Create a simbolic link (outext –> inext). The file names are implicitly given by the ABINIT file extension.

Example

outdir.symlink_abiext(‘1WF’, ‘DDK’)

creates the link out_DDK that points to out_1WF

Return: 0 if success.

Raise: RuntimeError

rename_abiext(inext, outext)

Rename the Abinit file with extension inext with the new extension outext

copy_abiext(inext, outext)

Copy the Abinit file with extension inext to a new file with the extension outext

remove_exts(exts)

Remove the files with the given extensions. Unlike rmtree, this function preserves the directory path. Return list with the absolute paths of the files that have been removed.

find_last_timden_file()

ABINIT produces lots of out_TIM1_DEN files for each step and we need to find the lat one in order to prepare the restart or to connect other tasks to the structural relaxation.

This function finds all the TIM?_DEN files in self and return a namedtuple (path, step) where path is the path of the last TIM?_DEN file and step is the iteration number. Returns None if the directory does not contain TIM?_DEN files.

find_1wf_files()

Abinit adds the idir-ipert index at the end of the 1WF file and this breaks the extension e.g. out_1WF4. This method scans the files in the directories and returns a list of namedtuple Each named tuple gives the path of the 1FK file and the pertcase index.

find_1den_files()

Abinit adds the idir-ipert index at the end of the 1DEN file and this breaks the extension e.g. out_DEN1. This method scans the files in the directories and returns a list of namedtuple Each named tuple gives the path of the 1DEN file and the pertcase index.

abipy.flowtk.utils.irdvars_for_ext(ext)

Returns a dictionary with the ABINIT variables that must be used to read the file with extension ext.

abipy.flowtk.utils.abi_extensions()

List with all the ABINIT extensions that are registered.

abipy.flowtk.utils.abi_splitext(filename)

Split the ABINIT extension from a filename. “Extension” are found by searching in an internal database.

Returns “(root, ext)” where ext is the registered ABINIT extension The final “.nc” is included (if any)

>>> assert abi_splitext("foo_WFK") == ('foo_', 'WFK')
>>> assert abi_splitext("/home/guido/foo_bar_WFK.nc") == ('foo_bar_', 'WFK.nc')

class abipy.flowtk.utils.FilepathFixer

Bases: object

This object modifies the names of particular output files produced by ABINIT so that the file extension is preserved. Having a one-to-one mapping between file extension and data format is indeed fundamental for the correct behaviour of abinit since:

• We locate the output file by just inspecting the file extension

• We select the variables that must be added to the input file on the basis of the extension specified by the user during the initialization of the AbinitFlow.

Unfortunately, ABINIT developers like to append extra stuff to the initial extension and therefore we have to call FilepathFixer to fix the output files produced by the run.

Example

fixer = FilepathFixer() fixer.fix_paths(‘/foo/out_1WF17’) == {‘/foo/out_1WF17’: ‘/foo/out_1WF’} fixer.fix_paths(‘/foo/out_1WF5.nc’) == {‘/foo/out_1WF5.nc’: ‘/foo/out_1WF.nc’}

fix_paths(paths)

Fix the filenames in the iterable paths

Returns

Mapping old_path –> new_path

Return type

old2new

abipy.flowtk.utils.map2rpn(map, obj)

Convert a Mongodb-like dictionary to a RPN list of operands and operators.

Reverse Polish notation (RPN) is a mathematical notation in which every operator follows all of its operands, e.g.

3 - 4 + 5 –> 3 4 - 5 +

>>> d = {2.0: {'$eq': 1.0}}
>>> assert map2rpn(d, None) == [2.0, 1.0, '$eq']

abipy.flowtk.utils.evaluate_rpn(rpn)

Evaluates the RPN form produced my map2rpn.

Returns: bool

class abipy.flowtk.utils.Condition(cmap=None)

Bases: object

This object receives a dictionary that defines a boolean condition whose syntax is similar to the one used in mongodb (albeit not all the operators available in mongodb are supported here).

Example:

$gt: {field: {$gt: value} }

$gt selects those documents where the value of the field is greater than (i.e. >) the specified value.

$and performs a logical AND operation on an array of two or more expressions (e.g. <expression1>, <expression2>, etc.) and selects the documents that satisfy all the expressions in the array.

{ $and: [ { <expression1> }, { <expression2> } , … , { <expressionN> } ] }

Consider the following example:

db.inventory.find( { qty: { $gt: 20 } } ) This query will select all documents in the inventory collection where the qty field value is greater than 20. Consider the following example:

db.inventory.find( { qty: { $gt: 20 } } ) db.inventory.find({ $and: [ { price: 1.99 }, { qty: { $lt: 20 } }, { sale: true } ] } )

classmethod as_condition(obj)

Convert obj into Condition

class abipy.flowtk.utils.Editor(editor=None)

Bases: object

Wrapper class that calls the editor specified by the user or the one specified in the $EDITOR env variable.

edit_files(fnames, ask_for_exit=True)

edit_file(fname)

static user_wants_to_exit()

Show an interactive prompt asking if exit is wanted.

class abipy.flowtk.utils.SparseHistogram(items, key=None, num=None, step=None)

Bases: object

plot(ax=None, **kwargs)

Plot the histogram with matplotlib, returns matplotlib figure.

Keyword arguments controlling the display of the figure:

kwargs	Meaning
title	Title of the plot (Default: None).
show	True to show the figure (default: True).
savefig	“abc.png” or “abc.eps” to save the figure to a file.
size_kwargs	Dictionary with options passed to fig.set_size_inches e.g. size_kwargs=dict(w=3, h=4)
tight_layout	True to call fig.tight_layout (default: False)
ax_grid	True (False) to add (remove) grid from all axes in fig. Default: None i.e. fig is left unchanged.
ax_annotate	Add labels to subplots e.g. (a), (b). Default: False
fig_close	Close figure. Default: False.

class abipy.flowtk.utils.Dirviz(top)

Bases: object

get_cluster_graph(engine='fdp', graph_attr=None, node_attr=None, edge_attr=None)

Generate directory graph in the DOT language. Directories are shown as clusters

Warning

This function scans the entire directory tree starting from top so the resulting graph can be really big.

Parameters

• engine – Layout command used. [‘dot’, ‘neato’, ‘twopi’, ‘circo’, ‘fdp’, ‘sfdp’, ‘patchwork’, ‘osage’]

• graph_attr – Mapping of (attribute, value) pairs for the graph.

• node_attr – Mapping of (attribute, value) pairs set for all nodes.

• edge_attr – Mapping of (attribute, value) pairs set for all edges.

Returns: graphviz.Digraph <https://graphviz.readthedocs.io/en/stable/api.html#digraph>

works Module

Works for Abinit

class abipy.flowtk.works.Work(workdir=None, manager=None)

Bases: abipy.flowtk.works.BaseWork, abipy.flowtk.works.NodeContainer

A Work is a list of (possibly connected) tasks.

Inheritance Diagram

set_manager(manager)

Set the abipy.flowtk.tasks.TaskManager to use to launch the abipy.flowtk.tasks.Task.

property flow

The flow containing this abipy.flowtk.works.Work.

set_flow(flow)

Set the flow associated to this abipy.flowtk.works.Work.

pos()

The position of self in the abipy.flowtk.flows.Flow

property pos_str

String representation of self.pos

set_workdir(workdir, chroot=False)

Set the working directory. Cannot be set more than once unless chroot is True

chroot(new_workdir)

postpone_on_all_ok()

This method should be called when additional tasks are added to the Work at runtime. e.g inside an on_all_ok method. This call triggers the allocations of new tasks so that the scheduler will continue execution since new tasks have been added and the finalized flag is set to False.

chunks(chunk_size)

Yield successive chunks of tasks of lenght chunk_size.

ipath_from_ext(ext)

Returns the path of the output file with extension ext. Use it when the file does not exist yet.

opath_from_ext(ext)

Returns the path of the output file with extension ext. Use it when the file does not exist yet.

property processes

Return a list of objects that support the subprocess.Popen protocol.

property all_done

True if all the abipy.flowtk.tasks.Task objects in the abipy.flowtk.works.Work are done.

property isnc

True if norm-conserving calculation.

property ispaw

True if PAW calculation.

property status_counter

Returns a Counter object that counts the number of task with given status (use the string representation of the status as key).

allocate(manager=None)

This function is called once we have completed the initialization of the abipy.flowtk.works.Work. It sets the manager of each task (if not already done) and defines the working directories of the tasks.

Parameters

manager – abipy.flowtk.tasks.TaskManager object or None

register(obj, deps=None, required_files=None, manager=None, task_class=None)

Registers a new abipy.flowtk.tasks.Task and add it to the internal list, taking into account possible dependencies.

Parameters

• obj – abipy.abio.inputs.AbinitInput instance or abipy.flowtk.tasks.Task object.

• deps – Dictionary specifying the dependency of this node or list of dependencies None means that this obj has no dependency.

• required_files – List of strings with the path of the files used by the task. Note that the files must exist when the task is registered. Use the standard approach based on Works, Tasks and deps if the files will be produced in the future.

• manager – The abipy.flowtk.tasks.TaskManager responsible for the submission of the task. If manager is None, we use the TaskManager specified during the creation of the abipy.flowtk.works.Work.

• task_class – Task subclass to instantiate. Default: AbinitTask

Returns: abipy.flowtk.tasks.Task object

register_task(obj, deps=None, required_files=None, manager=None, task_class=None)

Registers a new abipy.flowtk.tasks.Task and add it to the internal list, taking into account possible dependencies.

Parameters

• obj – abipy.abio.inputs.AbinitInput instance or abipy.flowtk.tasks.Task object.

• deps – Dictionary specifying the dependency of this node or list of dependencies None means that this obj has no dependency.

• required_files – List of strings with the path of the files used by the task. Note that the files must exist when the task is registered. Use the standard approach based on Works, Tasks and deps if the files will be produced in the future.

• manager – The abipy.flowtk.tasks.TaskManager responsible for the submission of the task. If manager is None, we use the TaskManager specified during the creation of the abipy.flowtk.works.Work.

• task_class – Task subclass to instantiate. Default: AbinitTask

Returns: abipy.flowtk.tasks.Task object

path_in_workdir(filename)

Create the absolute path of filename in the working directory.

setup(*args, **kwargs)

Method called before running the calculations. The default implementation is empty.

build(*args, **kwargs)

Creates the top level directory.

property status

Returns the status of the work i.e. the minimum of the status of the tasks.

get_all_status(only_min=False)

Returns a list with the status of the tasks in self.

Parameters

only_min – If True, the minimum of the status is returned.

check_status()

Check the status of the tasks.

get_panel(**kwargs)

Build panel with widgets to interact with the Work either in a notebook or in panel app. This is the implementation provided by the base class. Subclasses may provide specialized implementations.

rmtree(exclude_wildcard='')

Remove all files and directories in the working directory

Parameters

exclude_wildcard – Optional string with regular expressions separated by |. Files matching one of the regular expressions will be preserved. example: exclude_wildard=”.nc|.txt” preserves all the files whose extension is in [“nc”, “txt”].

rm_indatadir()

Remove all the indata directories.

rm_outdatadir()

Remove all the indata directories.

rm_tmpdatadir()

Remove all the tmpdata directories.

move(dest, isabspath=False)

Recursively move self.workdir to another location. This is similar to the Unix “mv” command. The destination path must not already exist. If the destination already exists but is not a directory, it may be overwritten depending on os.rename() semantics.

Be default, dest is located in the parent directory of self.workdir, use isabspath=True to specify an absolute path.

submit_tasks(wait=False)

Submits the task in self and wait. TODO: change name.

start(*args, **kwargs)

Start the work. Calls build and _setup first, then submit the tasks. Non-blocking call unless wait is set to True

read_etotals(unit='Ha')

Reads the total energy from the GSR file produced by the task.

Return a numpy array with the total energies in Hartree The array element is set to np.inf if an exception is raised while reading the GSR file.

parse_timers()

Parse the TIMER section reported in the ABINIT output files.

Returns

AbinitTimerParser object

class abipy.flowtk.works.BandStructureWork(scf_input, nscf_input, dos_inputs=None, workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

Work for band structure calculations.

Inheritance Diagram

plot_ebands(**kwargs)

Plot the band structure. kwargs are passed to the plot method of abipy.electrons.ebands.ElectronBands.

Return: matplotlib.figure.Figure

plot_ebands_with_edos(dos_pos=0, method='gaussian', step=0.01, width=0.1, **kwargs)

Plot the band structure and the DOS.

Parameters

• dos_pos – Index of the task from which the DOS should be obtained (note: 0 refers to the first DOS task).

• method – String defining the method for the computation of the DOS.

• step – Energy step (eV) of the linear mesh.

• width – Standard deviation (eV) of the gaussian.

• kwargs – Keyword arguments passed to plot_with_edos method to customize the plot.

Return: matplotlib.figure.Figure

plot_edoses(dos_pos=None, method='gaussian', step=0.01, width=0.1, **kwargs)

Plot the band structure and the DOS.

Parameters

• dos_pos – Index of the task from which the DOS should be obtained. None is all DOSes should be displayed. Accepts integer or list of integers.

• method – String defining the method for the computation of the DOS.

• step – Energy step (eV) of the linear mesh.

• width – Standard deviation (eV) of the gaussian.

• kwargs – Keyword arguments passed to plot method to customize the plot.

Return: matplotlib.figure.Figure

class abipy.flowtk.works.RelaxWork(ion_input, ioncell_input, workdir=None, manager=None, target_dilatmx=None)

Bases: abipy.flowtk.works.Work

Work for structural relaxations. The first task relaxes the atomic position while keeping the unit cell parameters fixed. The second task uses the final structure to perform a structural relaxation in which both the atomic positions and the lattice parameters are optimized.

Inheritance Diagram

on_ok(sender)

This callback is called when one task reaches status S_OK. If sender == self.ion_task, we update the initial structure used by self.ioncell_task and we unlock it so that the job can be submitted.

plot_ion_relaxation(**kwargs)

Plot the history of the ion-cell relaxation. kwargs are passed to the plot method of abipy.dynamics.hist.HistFile

Return: matplotlib.figure.Figure or None if hist file is not found.

plot_ioncell_relaxation(**kwargs)

Plot the history of the ion-cell relaxation. kwargs are passed to the plot method of abipy.dynamics.hist.HistFile

Return: matplotlib.figure.Figure or None if hist file is not found.

class abipy.flowtk.works.G0W0Work(scf_inputs, nscf_inputs, scr_inputs, sigma_inputs, workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

Work for generic G0W0 calculations. All input can be either single inputs or lists of inputs

Inheritance Diagram

class abipy.flowtk.works.QptdmWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

This work parallelizes the calculation of the q-points of the screening. It also provides the callback on_all_ok that calls mrgscr to merge all the partial screening files produced.

Inheritance Diagram

create_tasks(**kwargs)

merge_scrfiles(remove_scrfiles=True)

This method is called when all the q-points have been computed. It runs mrgscr in sequential on the local machine to produce the final SCR file in the outdir of the Work. If remove_scrfiles is True, the partial SCR files are removed after the merge.

on_all_ok()

This method is called when all the q-points have been computed. It runs mrgscr in sequential on the local machine to produce the final SCR file in the outdir of the Work.

class abipy.flowtk.works.SigmaConvWork(wfk_node, scr_node, sigma_inputs, workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

Work for self-energy convergence studies.

Inheritance Diagram

class abipy.flowtk.works.BseMdfWork(scf_input, nscf_input, bse_inputs, workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

Work for simple BSE calculations in which the self-energy corrections are approximated by the scissors operator and the screening is modeled with the model dielectric function.

Inheritance Diagram

get_mdf_robot()

Builds and returns a MdfRobot for analyzing the results in the MDF files.

class abipy.flowtk.works.PhononWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work, abipy.flowtk.works.MergeDdb

This work consists of nirred Phonon tasks where nirred is the number of irreducible atomic perturbations for a given set of q-points. It provides the callback method (on_all_ok) that calls mrgddb (mrgdv) to merge all the partial DDB (POT) files produced. The two files are available in the output directory of the Work.

Inheritance Diagram

classmethod from_scf_task(scf_task, qpoints, is_ngqpt=False, with_becs=False, with_quad=False, with_flexoe=False, with_dvdb=True, tolerance=None, ddk_tolerance=None, prtwf=- 1, manager=None)

Construct a PhononWork from a abipy.flowtk.tasks.ScfTask object. The input file for phonons is automatically generated from the input of the ScfTask. Each phonon task depends on the WFK file produced by the scf_task.

Parameters

• scf_task – abipy.flowtk.tasks.ScfTask object.

• qpoints – q-points in reduced coordinates. Accepts single q-point, list of q-points or three integers defining the q-mesh if is_ngqpt.

• is_ngqpt – True if qpoints should be interpreted as divisions instead of q-points.

• with_becs – Activate calculation of Electric field and Born effective charges.

• with_quad – Activate calculation of dynamical quadrupoles. Require with_becs Note that only selected features are compatible with dynamical quadrupoles. Please consult <https://docs.abinit.org/topics/longwave/>

• with_flexoe – True to activate computation of flexoelectric tensor. Require with_becs

• with_dvdb – True to merge POT1 files associated to atomic perturbations in the DVDB file at the end of the calculation

• tolerance – dict {“varname”: value} with the tolerance to be used in the phonon run. None to use AbiPy default.

• ddk_tolerance – dict {“varname”: value} with the tolerance used in the DDK run if with_becs. None to use AbiPy default.

• prtwf – Controls the output of the first-order WFK. By default we set it to -1 when q != 0 so that AbiPy is still able to restart the DFPT task if the calculation is not converged (worst case scenario) but we avoid the output of the 1-st WFK if the calculation converged successfully. Non-linear DFT applications should not be affected since they assume q == 0.

• manager – abipy.flowtk.tasks.TaskManager object.

classmethod from_scf_input(scf_input, qpoints, is_ngqpt=False, with_becs=False, with_quad=False, with_flexoe=False, with_dvdb=True, tolerance=None, ddk_tolerance=None, prtwf=- 1, manager=None)

Similar to from_scf_task, the difference is that this method requires an input for SCF calculation. A new abipy.flowtk.tasks.ScfTask is created and added to the Work. This API should be used if the DDB of the GS task should be merged.

on_all_ok()

This method is called when all the q-points have been computed. It runs mrgddb in sequential on the local machine to produce the final DDB file in the outdir of the abipy.flowtk.works.Work.

class abipy.flowtk.works.PhononWfkqWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work, abipy.flowtk.works.MergeDdb

This work computes phonons with DFPT on an arbitrary q-mesh (usually denser than the k-mesh for electrons) by computing WKQ files for each q-point. The number of irreducible atomic perturbations for each q-point is taken into account. It provides the callback method (on_all_ok) that calls mrgddb (mrgdv) to merge all the partial DDB (POT) files produced. The two files are available in the output directory of the Work. The WKQ files are removed at runtime.

Inheritance Diagram

classmethod from_scf_task(scf_task, ngqpt, ph_tolerance=None, tolwfr=1e-22, nband=None, with_becs=False, ddk_tolerance=None, shiftq=(0, 0, 0), is_ngqpt=True, remove_wfkq=True, prepgkk=0, manager=None)

Construct a PhononWfkqWork from a abipy.flowtk.tasks.ScfTask object. The input files for WFQ and phonons are automatically generated from the input of the ScfTask. Each phonon task depends on the WFK file produced by scf_task and the associated WFQ file.

Parameters

• scf_task – abipy.flowtk.tasks.ScfTask object.

• ngqpt – three integers defining the q-mesh

• with_becs – Activate calculation of Electric field and Born effective charges.

• ph_tolerance – dict {“varname”: value} with the tolerance for the phonon run. None to use AbiPy default.

• tolwfr – tolerance used to compute WFQ.

• ddk_tolerance – dict {“varname”: value} with the tolerance used in the DDK run if with_becs. None to use AbiPy default.

• shiftq – Q-mesh shift. Multiple shifts are not supported.

• is_ngqpt – the ngqpt is interpreted as a set of integers defining the q-mesh, otherwise is an explicit list of q-points

• remove_wfkq – Remove WKQ files when the children are completed.

• prepgkk – 1 to activate computation of all 3*natom perts (debugging option).

• manager – abipy.flowtk.tasks.TaskManager object.

on_ok(sender)

This callback is called when one task reaches status S_OK. It removes the WFKQ file if all its children have reached S_OK.

on_all_ok()

This method is called when all the q-points have been computed. Ir runs mrgddb in sequential on the local machine to produce the final DDB file in the outdir of the abipy.flowtk.works.Work.

class abipy.flowtk.works.GKKPWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

This work computes electron-phonon matrix elements for all the q-points present in a DVDB and DDB file

Inheritance Diagram

classmethod from_den_ddb_dvdb(inp, den_path, ddb_path, dvdb_path, mpiprocs=1, remove_wfkq=True, qpath=None, with_ddk=True, expand=True, manager=None)

Construct a PhononWfkqWork from a DDB and DVDB file. For each q found, a WFQ task and an EPH task computing the matrix elements are created.

classmethod from_phononwfkq_work(phononwfkq_work, nscf_vars={}, remove_wfkq=True, with_ddk=True, manager=None)

Construct a GKKPWork from a PhononWfkqWork object. The WFQ are the ones used for PhononWfkqWork so in principle have only valence bands

on_ok(sender)

This callback is called when one task reaches status S_OK. It removes the WFKQ file if all its children have reached S_OK.

class abipy.flowtk.works.BecWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work, abipy.flowtk.works.MergeDdb

Work for the computation of the Born effective charges.

This work consists of DDK tasks and phonon + electric field perturbation It provides the callback method (on_all_ok) that calls mrgddb to merge the partial DDB files produced by the work.

Inheritance Diagram

classmethod from_scf_task(scf_task, ddk_tolerance=None, ph_tolerance=None, manager=None)

Build tasks for the computation of Born effective charges from a ground-state task.

Parameters

• scf_task – abipy.flowtk.tasks.ScfTask object.

• ddk_tolerance – tolerance used in the DDK run if with_becs. None to use AbiPy default.

• ph_tolerance – dict {“varname”: value} with the tolerance used in the phonon run. None to use AbiPy default.

• manager – abipy.flowtk.tasks.TaskManager object.

on_all_ok()

This method is called when all tasks reach S_OK Ir runs mrgddb in sequential on the local machine to produce the final DDB file in the outdir of the abipy.flowtk.works.Work.

class abipy.flowtk.works.DteWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work, abipy.flowtk.works.MergeDdb

Work for the computation of the third derivative of the energy.

This work consists of DDK tasks and electric field perturbation. It provides the callback method (on_all_ok) that calls mrgddb to merge the partial DDB files produced

Inheritance Diagram

classmethod from_scf_task(scf_task, ddk_tolerance=None, manager=None)

Build a DteWork from a ground-state task.

Parameters

• scf_task – abipy.flowtk.tasks.ScfTask object.

• ddk_tolerance – tolerance used in the DDK run if with_becs. None to use AbiPy default.

• manager – abipy.flowtk.tasks.TaskManager object.

on_all_ok()

This method is called when all tasks reach S_OK It runs mrgddb in sequential on the local machine to produce the final DDB file in the outdir of the Work.

class abipy.flowtk.works.ConducWork(workdir=None, manager=None)

Bases: abipy.flowtk.works.Work

Workflow for the computation of electrical conductivity.

Can be called from:

1. MultiDataset and PhononWork

2. MultiDataset, DDB filepath and DVDB filepath.

Can use Kerange Capability using withKerange=True

This work consists of 3 tasks or 5 tasks with kerange:

1. SCF GS

2. NSCF

3. Kerange (Kerange only)

4. WFK Interpolation (Kerange only)

5. Electrical Conductivity Calculation.

classmethod from_phwork(phwork, multi, nbr_proc=None, flow=None, with_kerange=False, omp_nbr_thread=1, manager=None)

Construct a ConducWork from a abipy.flowtk.works.PhononWork and abipy.abio.inputs.MultiDataset.

Parameters

• phwork – abipy.flowtk.works.PhononWork object calculating the DDB and DVDB files.

• multi – abipy.abio.inputs.MultiDataset object containing a list of 3 datasets or 5 with Kerange. See abipy/abio/factories.py -> conduc_from_scf_nscf_inputs for details about multi.

• nbr_proc – Required if with_kerange since autoparal doesn’t work with optdriver=8.

• flow – The flow calling the work. Used for with_fixed_mpi_omp.

• with_kerange – True if using Kerange.

• omp_nbr_thread – Number of omp_thread to use.

• manager – abipy.flowtk.tasks.TaskManager of the task. If None, the manager is initialized from the config file.

classmethod from_filepath(ddb_path, dvdb_path, multi, nbr_proc=None, flow=None, with_kerange=False, omp_nbr_thread=1, manager=None)

Construct a ConducWork from previously calculated DDB/DVDB file and abipy.abio.inputs.MultiDataset.

Parameters

• multi – a abipy.abio.inputs.MultiDataset object containing a list of 3 datasets or 5 with Kerange. See abipy/abio/factories.py -> conduc_from_scf_nscf_inputs for details about multi.

• ddb_path – a string containing the path to the DDB file.

• dvdb_path – a string containing the path to the DVDB file.

• nbr_proc – Required if with_kerange since autoparal doesn’t work with optdriver=8.

• flow – The flow calling the work, only needed if with_kerange.

• with_kerange – True if using Kerange.

• omp_nbr_thread – Number of omp_thread to use.

• manager – abipy.flowtk.tasks.TaskManager of the task. If None, the manager is initialized from the config file.

wrappers Module

Wrappers for ABINIT main executables

class abipy.flowtk.wrappers.Mrgscr(manager=None, executable=None, verbose=0)

Bases: abipy.flowtk.wrappers.ExecWrapper

merge_qpoints(workdir, files_to_merge, out_prefix)

Execute mrgscr inside directory workdir to merge files_to_merge. Produce new file with prefix out_prefix

class abipy.flowtk.wrappers.Mrgddb(manager=None, executable=None, verbose=0)

Bases: abipy.flowtk.wrappers.ExecWrapper

merge(workdir, ddb_files, out_ddb, description, delete_source_ddbs=True)

Merge DDB file, return the absolute path of the new database in workdir.

class abipy.flowtk.wrappers.Mrgdvdb(manager=None, executable=None, verbose=0)

Bases: abipy.flowtk.wrappers.ExecWrapper

merge(workdir, pot_files, out_dvdb, delete_source=True)

Merge POT files containing 1st order DFPT potential return the absolute path of the new database in workdir.

Parameters

delete_source – True if POT1 files should be removed after (successful) merge.