abiflows.fireworks.utils package

Subpackages

Submodules

abiflows.fireworks.utils.custodian_utils module

class abiflows.fireworks.utils.custodian_utils.SRCErrorHandler[source]

Bases: custodian.custodian.ErrorHandler

HANDLER_PRIORITIES = {'PRIORITY_FIRST': 0, 'PRIORITY_HIGH': 2, 'PRIORITY_LAST': 6, 'PRIORITY_LOW': 4, 'PRIORITY_MEDIUM': 3, 'PRIORITY_VERY_HIGH': 1, 'PRIORITY_VERY_LOW': 5}
PRIORITY_FIRST = 0
PRIORITY_VERY_HIGH = 1
PRIORITY_HIGH = 2
PRIORITY_MEDIUM = 3
PRIORITY_LOW = 4
PRIORITY_VERY_LOW = 5
PRIORITY_LAST = 6
abstract as_dict()[source]

A JSON serializable dict representation of an object.

abstract from_dict(d)[source]
Parameters

d – Dict representation.

Returns

MSONable class.

abstract setup()[source]
set_fw_spec(fw_spec)[source]
set_fw_to_check(fw_to_check)[source]
src_setup(fw_spec, fw_to_check)[source]
abstract property handler_priority
property skip_remaining_handlers
abstract property allow_fizzled
abstract property allow_completed
abstract has_corrections()[source]
class abiflows.fireworks.utils.custodian_utils.MonitoringSRCErrorHandler[source]

Bases: custodian.custodian.ErrorHandler

HANDLER_PRIORITIES = {'PRIORITY_FIRST': 0, 'PRIORITY_HIGH': 2, 'PRIORITY_LAST': 6, 'PRIORITY_LOW': 4, 'PRIORITY_MEDIUM': 3, 'PRIORITY_VERY_HIGH': 1, 'PRIORITY_VERY_LOW': 5}
PRIORITY_FIRST = 0
PRIORITY_VERY_HIGH = 1
PRIORITY_HIGH = 2
PRIORITY_MEDIUM = 3
PRIORITY_LOW = 4
PRIORITY_VERY_LOW = 5
PRIORITY_LAST = 6
abstract as_dict()[source]

A JSON serializable dict representation of an object.

abstract from_dict(d)[source]
Parameters

d – Dict representation.

Returns

MSONable class.

abstract property handler_priority
property skip_remaining_handlers
class abiflows.fireworks.utils.custodian_utils.SRCValidator[source]

Bases: custodian.custodian.Validator

HANDLER_PRIORITIES = {'PRIORITY_FIRST': 0, 'PRIORITY_HIGH': 2, 'PRIORITY_LAST': 6, 'PRIORITY_LOW': 4, 'PRIORITY_MEDIUM': 3, 'PRIORITY_VERY_HIGH': 1, 'PRIORITY_VERY_LOW': 5}
PRIORITY_FIRST = 0
PRIORITY_VERY_HIGH = 1
PRIORITY_HIGH = 2
PRIORITY_MEDIUM = 3
PRIORITY_LOW = 4
PRIORITY_VERY_LOW = 5
PRIORITY_LAST = 6

abiflows.fireworks.utils.databases module

Utilities for database insertion

class abiflows.fireworks.utils.databases.MongoDatabase(host, port, database, username, password, collection, gridfs_collection=None)[source]

Bases: monty.json.MSONable

MongoDB database class for access, insertion, update, … in a MongoDB database

insert_entry(entry, gridfs_msonables=None)[source]
get_entry(criteria)[source]
save_entry(entry)[source]
update_entry(query, entry_update, gridfs_msonables=None)[source]
as_dict()[source]

Json-serializable dict representation of a MongoDatabase

classmethod from_dict(d)[source]
Parameters

d – Dict representation.

Returns

MSONable class.

class abiflows.fireworks.utils.databases.StorageServer(hostname, port=22, username=None, password=None, server_type='REMOTE_SERVER')[source]

Bases: monty.json.MSONable

Storage server class for moving files to/from a given server

REMOTE_SERVER = 'REMOTE_SERVER'
LOCAL_SERVER = 'LOCAL_SERVER'
connect()[source]
disconnect()[source]
remotepath_exists(path)[source]
remote_makedirs(path)[source]
put(localpath, remotepath, overwrite=False, makedirs=True)[source]
get(remotepath, localpath=None, overwrite=False, makedirs=True)[source]
as_dict()[source]

Json-serializable dict representation of a StorageServer

classmethod from_dict(d)[source]
Parameters

d – Dict representation.

Returns

MSONable class.

abiflows.fireworks.utils.fw_utils module

Utilities for fireworks

abiflows.fireworks.utils.fw_utils.append_fw_to_wf(new_fw, wf)[source]
abiflows.fireworks.utils.fw_utils.get_short_single_core_spec(fw_manager=None, master_mem_overhead=0, return_qtk=False, timelimit=None)[source]
abiflows.fireworks.utils.fw_utils.set_short_single_core_to_spec(spec={}, master_mem_overhead=0, fw_manager=None)[source]
class abiflows.fireworks.utils.fw_utils.FWTaskManager(**kwargs)[source]

Bases: object

Object containing the configuration parameters and policies to run abipy. The policies needed for the abinit FW will always be available through default values. These can be overridden also setting the parameters in the spec. The standard abipy task manager is contained as an object on its own that can be used to run the autoparal or factories if needed. The rationale behind this choice, instead of subclassing, is to not force the user to fill the qadapter part of the task manager, which is needed only for the autoparal, but is required in the TaskManager initialization. Wherever the TaskManager is needed just pass the ftm.task_manager. The TaskManager part can be loaded from an external manager.yml file using the “abipy_manager” key in fw_policy. This is now the preferred choice. If this value is not defined, it will be loaded with TaskManager.from_user_config

YAML_FILE = 'fw_manager.yaml'
USER_CONFIG_DIR = '/Users/gmatteo/.abinit/abipy'
fw_policy_defaults = {'abinit_cmd': 'abinit', 'abipy_manager': None, 'allow_local_restart': False, 'anaddb_cmd': 'anaddb', 'autoparal': False, 'continue_unconverged_on_rerun': True, 'copy_deps': False, 'cut3d_cmd': 'cut3d', 'max_restarts': 10, 'mpirun_cmd': 'mpirun', 'mrgddb_cmd': 'mrgddb', 'recover_previous_job': True, 'rerun_same_dir': False, 'short_job_timelimit': 600, 'timelimit_buffer': 120, 'walltime_command': None}
class FWPolicy(rerun_same_dir, max_restarts, autoparal, abinit_cmd, mrgddb_cmd, anaddb_cmd, cut3d_cmd, mpirun_cmd, copy_deps, walltime_command, continue_unconverged_on_rerun, allow_local_restart, timelimit_buffer, short_job_timelimit, recover_previous_job, abipy_manager)

Bases: tuple

property abinit_cmd

Alias for field number 3

property abipy_manager

Alias for field number 15

property allow_local_restart

Alias for field number 11

property anaddb_cmd

Alias for field number 5

property autoparal

Alias for field number 2

property continue_unconverged_on_rerun

Alias for field number 10

property copy_deps

Alias for field number 8

property cut3d_cmd

Alias for field number 6

property max_restarts

Alias for field number 1

property mpirun_cmd

Alias for field number 7

property mrgddb_cmd

Alias for field number 4

property recover_previous_job

Alias for field number 14

property rerun_same_dir

Alias for field number 0

property short_job_timelimit

Alias for field number 13

property timelimit_buffer

Alias for field number 12

property walltime_command

Alias for field number 9

classmethod from_user_config(fw_policy=None)[source]

Initialize the manager using the dict in the following order of preference: - the “fw_manager.yaml” file in the folder where the command is executed - a yaml file pointed by the “FW_TASK_MANAGER” - the “fw_manager.yaml” in the ~/.abinit/abipy folder - if no file available, fall back to default values

classmethod from_file(path)[source]

Read the configuration parameters from the Yaml file filename.

has_task_manager()[source]
update_fw_policy(d)[source]
abiflows.fireworks.utils.fw_utils.get_time_report_for_wf(wf)[source]
abiflows.fireworks.utils.fw_utils.get_last_completed_launch(fw)[source]

Given a Firework object returns the last completed launch

abiflows.fireworks.utils.fw_utils.load_abitask(fw)[source]

Given a Firework object returns the abinit related task contained. Sets the list of directories set from the last completed launch. If no abinit related firetasks are found or the task has no completed launch returns None.

abiflows.fireworks.utils.fw_utils.get_fw_by_task_index(wf, task_tag, index=1)[source]

Given a workflow object (with connection to the db) returns the wf corresponding to the task_type.

Parameters

  • wf – a fireworks Workflow object.

  • task_tag – the task tag associated with the task as defined in abinit_workflows. Should not include the index.

  • index – the numerical or text index of the task. If negative the the last fw corresponding to task_tag will be selected. If None, no index will be considered and the first match will be returned.

Returns

a fireworks Firework object. None if no match is found.

abiflows.fireworks.utils.fw_utils.get_lp_and_fw_id_from_task(task, fw_spec)[source]

Given an instance of a running task and its spec, tries to load the LaunchPad and the current fw_id. It will first check for “_add_launchpad_and_fw_id”, then try to load from FW.json/FW.yaml file.

Should be used inside tasks that require to access to the LaunchPad and to the whole workflow. :param task: An instance of a running task :param fw_spec: The spec of the task

Returns

an instance of LaunchPah and the fw_id of the current task

abiflows.fireworks.utils.math_utils module

abiflows.fireworks.utils.math_utils.prime_factors(n)[source]

Lists prime factors of a given natural integer, from greatest to smallest :param n: Natural integer :rtype : list of all prime factors of the given natural n

abiflows.fireworks.utils.math_utils.divisors(n)[source]

From a given natural integer, returns the list of divisors in ascending order :param n: Natural integer :return: List of divisors of n in ascending order

abiflows.fireworks.utils.task_history module

Task history related objects

class abiflows.fireworks.utils.task_history.TaskHistory[source]

Bases: collections.deque, monty.json.MSONable

History class for tracking the creation and actions performed during a task. The objects provided should be PGMSONable, thus dicts, lists or PGMSONable objects. The expected items are dictionaries, transformations, corrections, autoparal, restart/reset and initializations (when factories will be handled). Possibly, the first item should contain information about the starting point of the task. This object will be forwarded during task restarts and resets, in order to keep track of the full history of the task.

as_dict()[source]

A JSON serializable dict representation of an object.

classmethod from_dict(d)[source]
Parameters

d – Dict representation.

Returns

MSONable class.

log_initialization(task, initialization_info=None)[source]
log_corrections(corrections)[source]
log_restart(restart_info, local_restart=False)[source]
log_autoparal(optconf)[source]
log_unconverged()[source]
log_finalized(final_input=None)[source]
log_converge_params(unconverged_params, abiinput)[source]
log_error(exc)[source]
log_abinit_stop(run_time=None)[source]
get_events_by_types(types)[source]

Return the events in history of the selected types. types can be a single type or a list

get_total_run_time()[source]

Calculates total run time based summing the run times saved in the abinit stop event.

class abiflows.fireworks.utils.task_history.TaskEvent(event_type, details=None)[source]

Bases: monty.json.MSONable

Object used to categorize the events in the TaskHistory.

INITIALIZED = 'initialized'
CORRECTIONS = 'corrections'
RESTART = 'restart'
AUTOPARAL = 'autoparal'
UNCONVERGED = 'unconverged'
FINALIZED = 'finalized'
UNCONVERGED_PARAMS = 'unconverged parameters'
ERROR = 'error'
ABINIT_STOP = 'abinit stop'
as_dict()[source]

A JSON serializable dict representation of an object.

classmethod from_dict(d)[source]
Parameters

d – Dict representation.

Returns

MSONable class.

abiflows.fireworks.utils.time_utils module

abiflows.fireworks.utils.time_utils.seconds_to_hms(seconds)[source]

Converts second to the format “h:mm:ss”

Parameters

seconds – number of seconds

Returns

mm:ss”. An empty string if seconds is None.

Return type

A string representing the seconds with the format “h

class abiflows.fireworks.utils.time_utils.TimeReport(total_run_time, n_fws, total_cpu_time=None, contributed_cpu_time=0, total_run_time_per_tag=None, total_cpu_time_per_tag=None, contributed_cpu_time_per_tag=None, worker=None)[source]

Bases: monty.json.MSONable

Report of the time consumed for the whole workflow and for each single step. Includes both the time required for the calculations as well as the cpu time consumed.

as_dict()[source]

A JSON serializable dict representation of an object.

classmethod from_dict(d)[source]
Parameters

d – Dict representation.

Returns

MSONable class.

Module contents