api¶

Python ReproMan API exposing user-oriented commands (also available via CLI)

backend_parameters¶

reproman.api.backend_parameters(backends=None)¶: Display available backend parameters.

create¶

reproman.api.create(name, resource_type, backend_parameters)¶

Create a computation environment

Parameters:	name (str) – Name of the resource to create. Constraints: value must be a string. resource_type (str) – Resource type to create. Constraints: value must be a string. backend_parameters – One or more backend parameters in the form KEY=VALUE. Use the command reproman backend-parameters to see the list of available backend parameters.

delete¶

reproman.api.delete(resref, resref_type='auto', skip_confirmation=False, force=False)¶

Delete a computation environment

Examples

$ reproman delete my-resource

Parameters:

resref (str or None) – Name or ID of the resource to operate on. To see available resources, run ‘reproman ls’. Constraints: value must be a string, or value must be None.
resref_type ({auto, name, id}, optional) – A resource can be referenced by its name or ID. In the unlikely case that a name collides with an ID, explicitly specify ‘name’ or ‘id’ to disambiguate. Constraints: value must be one of (‘auto’, ‘name’, ‘id’). [Default: ‘auto’]
skip_confirmation (bool, optional) – Delete resource without prompting user for confirmation. [Default: False]
force (bool, optional) – Remove a resource from the local inventory regardless of connection errors. Use with caution!. [Default: False]

diff¶

reproman.api.diff(prov1, prov2, satisfies)¶

Report if a specification satisfies the requirements in another specification

Examples

$ reproman diff environment1.yml environment2.yml

Parameters:	prov1 (str) – ReproMan provenance file. Constraints: value must be a string. prov2 (str) – ReproMan provenance file. Constraints: value must be a string. satisfies (bool) – Make sure the first environment satisfies the needs of the second environment.

execute¶

reproman.api.execute(command, args, resref=None, resref_type='auto', internal=False, trace=False)¶

Execute a command in a computation environment

Examples

$ reproman execute mkdir /home/blah/data

Parameters:

command (str) – name of the command to run. Constraints: value must be a string.
args (str) – list of positional and keyword args to pass to the command. Constraints: list expected, each value must be a string.
resref (str or None, optional) – Name or ID of the resource to operate on. To see available resources, run ‘reproman ls’. Constraints: value must be a string, or value must be None. [Default: None]
resref_type ({auto, name, id}, optional) – A resource can be referenced by its name or ID. In the unlikely case that a name collides with an ID, explicitly specify ‘name’ or ‘id’ to disambiguate. Constraints: value must be one of (‘auto’, ‘name’, ‘id’). [Default: ‘auto’]
internal (bool, optional) – Instead of running a generic/any command, execute the internal ReproMan command available within sessions. Known are: mkdir, isdir, put, get, chown, chmod. [Default: False]
trace (bool, optional) – if set, trace execution within the environment. [Default: False]

install¶

reproman.api.install(resref, spec, resref_type='auto')¶

Install packages according to the provided specification(s)

Examples

$ reproman install docker recipe_for_failure.yml

Parameters:

resref (str or None) – Name or ID of the resource to operate on. To see available resources, run ‘reproman ls’. Constraints: value must be a string, or value must be None.
spec (str) – file with specifications (in supported formats) of packages used in executed environment. Constraints: list expected, each value must be a string.
resref_type ({auto, name, id}, optional) – A resource can be referenced by its name or ID. In the unlikely case that a name collides with an ID, explicitly specify ‘name’ or ‘id’ to disambiguate. Constraints: value must be one of (‘auto’, ‘name’, ‘id’). [Default: ‘auto’]

jobs¶

reproman.api.jobs(queries, action='auto', all_=False, status=False)¶

View and manage reproman run jobs.

The possible actions are

list: Display a oneline list of all registered jobs

show: Display more information for each job over multiple lines

delete: Unregister a job locally

fetch: Fetch a completed job

auto: If jobs are specified (via JOB or –all), behave like ‘fetch’. Otherwise, behave like ‘list’.

Parameters:

queries – A full job ID or a unique substring.
action ({auto, list, show, delete, fetch}, optional) – Operation to perform on the job(s). Constraints: value must be one of (‘auto’, ‘list’, ‘show’, ‘delete’, ‘fetch’). [Default: ‘auto’]
all (bool, optional) – Operate on all jobs. [Default: False]
status (bool, optional) – Query the resource for status information when listing or showing jobs. [Default: False]

login¶

reproman.api.login(resref, resref_type='auto')¶

Log into a computation environment

Examples

$ reproman login my-resource

Parameters:

resref (str or None) – Name or ID of the resource to operate on. To see available resources, run ‘reproman ls’. Constraints: value must be a string, or value must be None.
resref_type ({auto, name, id}, optional) – A resource can be referenced by its name or ID. In the unlikely case that a name collides with an ID, explicitly specify ‘name’ or ‘id’ to disambiguate. Constraints: value must be one of (‘auto’, ‘name’, ‘id’). [Default: ‘auto’]

ls¶

reproman.api.ls(resrefs=None, resref_type='auto', verbose=False, refresh=False)¶

List known computation resources, images and environments

Examples

$ reproman ls

Parameters:

resrefs – Restrict the output to this resource name or ID. [Default: None]
resref_type ({auto, name, id}, optional) – A resource can be referenced by its name or ID. In the unlikely case that a name collides with an ID, explicitly specify ‘name’ or ‘id’ to disambiguate. Constraints: value must be one of (‘auto’, ‘name’, ‘id’). [Default: ‘auto’]
verbose (bool, optional) – provide more verbose listing. [Default: False]
refresh (bool, optional) – Refresh the status of the resources listed. [Default: False]

retrace¶

reproman.api.retrace(path=None, spec=None, output_file=None, resref=None, resref_type='auto')¶

Gather detailed package information from paths or a ReproZip trace file.

Examples

$ reproman retrace –spec reprozip_run.yml > reproman_config.yml

Parameters:

path (str or None, optional) – path(s) to be traced. If spec is provided, would trace them after tracing the spec. Constraints: list expected, each value must be a string, or value must be None. [Default: None]
spec (str or None, optional) – ReproZip YML file to be analyzed. Constraints: value must be a string, or value must be None. [Default: None]
output_file (str or None, optional) – Output file. If not specified - printed to stdout. Constraints: value must be a string, or value must be None. [Default: None]
resref (str or None, optional) – Name or ID of the resource to operate on. To see available resources, run ‘reproman ls’.Note: As a special case, a session instance can be passed as the value for resref. . Constraints: value must be a string, or value must be None. [Default: None]
resref_type ({auto, name, id}, optional) – A resource can be referenced by its name or ID. In the unlikely case that a name collides with an ID, explicitly specify ‘name’ or ‘id’ to disambiguate. Constraints: value must be one of (‘auto’, ‘name’, ‘id’). [Default: ‘auto’]

run¶

reproman.api.run(command=None, message=None, resref=None, resref_type='auto', list_=None, submitter=None, orchestrator=None, batch_spec=None, batch_parameters=None, job_specs=None, job_parameters=None, inputs=None, outputs=None, follow=False)¶

Run a command on the specified resource.

Two main options control how the job is executed: the orchestator and the submitter. The orchestrator that is selected controls details like how the data is made available on the resource and how the results are fetched. The submitter controls how the job is submitted on the resource (e.g., as a condor job). Use –list to see information on the available orchestrators and submitters.

Unless –follow is specified, the job is started and detached. Use reproman jobs to list and fetch detached jobs.

Parameters:

command – command for execution. [Default: None]
message – Message to use when saving the run. The details depend on the orchestator, but in general this message will be used in the commit message. [Default: None]
resref (str or None, optional) – Name or ID of the resource to operate on. To see available resources, run ‘reproman ls’. Constraints: value must be a string, or value must be None. [Default: None]
resref_type ({auto, name, id}, optional) – A resource can be referenced by its name or ID. In the unlikely case that a name collides with an ID, explicitly specify ‘name’ or ‘id’ to disambiguate. Constraints: value must be one of (‘auto’, ‘name’, ‘id’). [Default: ‘auto’]
list – Show available submitters, orchestrators, or job parameters. If an empty string is given, show all. [Default: None]
submitter ({None, pbs, condor, slurm, local, lsf}, optional) – Name of submitter. The submitter controls how the command should be submitted on the resource (e.g., with condor_submit). Constraints: value must be one of (None, ‘pbs’, ‘condor’, ‘slurm’, ‘local’, ‘lsf’). [Default: None]
orchestrator ({None, plain, datalad-pair, datalad-no-remote, datalad-pair-run, datalad-local-run}, optional) – Name of orchestrator. The orchestrator performs pre- and post- command steps like setting up the directory for command execution and storing the results. Constraints: value must be one of (None, ‘plain’, ‘datalad-pair’, ‘datalad-no-remote’, ‘datalad-pair-run’, ‘datalad-local-run’). [Default: None]
batch_spec – YAML file that defines a series of records with parameters for commands. A command will be constructed for each record, with record values available in the command as well as the inputs and outputs as {p[KEY]}. See batch_parameters for an alternative method for simple combinations. [Default: None]
batch_parameters – Define batch parameters with ‘KEY=val1,val2,…’. Different keys can be specified by giving multiple values, in which case the product of the values are taken. For example, ‘subj=mei,satsuki’ and ‘day=1,2’ would expand to four records, pairing each subj with each day. Values can be a glob pattern to match against the current working directory. See batch_spec for specifying more complex records. . [Default: None]
job_specs – YAML files that define job parameters. Multiple paths can be given. If a parameter is defined in multiple specs, the value from the last path that defines it is used. . [Default: None]
job_parameters – A job parameter in the form KEY=VALUE. If the same parameter is defined via a job spec, the value given here takes precedence. The values are available as fields in the templates used to generate both the run script and submission script. . [Default: None]
inputs – An input path to the command. How input paths are used depends on the orchestrator, but, at the very least, the orchestrator should try to make these paths available on the resource. . [Default: None]
outputs – An output path to the command. How output paths are handled depends on the orchestrator. . [Default: None]
follow ({False, True, stop, stop-if-success, delete, delete-if-success}, optional) – Continue to follow the submitted command instead of submitting it and detaching. Constraints: value must be one of (False, True, ‘stop’, ‘stop-if-success’, ‘delete’, ‘delete-if-success’). [Default: False]

start¶

reproman.api.start(resref, resref_type='auto')¶

Start a computation environment

Examples

$ reproman start my-resource

Parameters:

resref (str or None) – Name or ID of the resource to operate on. To see available resources, run ‘reproman ls’. Constraints: value must be a string, or value must be None.
resref_type ({auto, name, id}, optional) – A resource can be referenced by its name or ID. In the unlikely case that a name collides with an ID, explicitly specify ‘name’ or ‘id’ to disambiguate. Constraints: value must be one of (‘auto’, ‘name’, ‘id’). [Default: ‘auto’]

stop¶

reproman.api.stop(resref, resref_type='auto')¶

Stop a computation environment

Examples

$ reproman stop my-resource

Parameters:

resref (str or None) – Name or ID of the resource to operate on. To see available resources, run ‘reproman ls’. Constraints: value must be a string, or value must be None.
resref_type ({auto, name, id}, optional) – A resource can be referenced by its name or ID. In the unlikely case that a name collides with an ID, explicitly specify ‘name’ or ‘id’ to disambiguate. Constraints: value must be one of (‘auto’, ‘name’, ‘id’). [Default: ‘auto’]

test¶

reproman.api.test()¶

Run internal ReproMan (unit)tests.

This can be used to verify correct operation on the system